2024-09-16T00:21:12.821 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T00:21:12.846 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:21:12.944 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906403' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.dir_split install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHCbibadsvHOL/gj/VPqzsA4NHKTi00ueduN8gAssw/kKP/z/F/jWYchlyqXt1bQsMQmmpfUfgTIFbMUCp+Z1HA= smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLg2Snc1G87T0ztM9lLZ9F56LLPw0xGGKStWI1voQ8UnmYK9CPNgD5uapE7WSyDFiIRQ5ctQZK5Ng/yIC6pkbXo= smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWfzkCbCKpSe1tlGzDnfAcLHYsVvEpZUAEkwAI9ZjQ0xJreDUUl9TcxylU2GjM+P3z28PZMLblr+Vkhrg3IbwE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T00:21:12.945 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-16T00:21:12.945 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-16T00:21:12.946 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T00:21:12.947 INFO:teuthology.task.internal:Checking packages... 2024-09-16T00:21:12.971 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-16T00:21:12.971 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:21:12.971 INFO:teuthology.packaging:ref: None 2024-09-16T00:21:12.971 INFO:teuthology.packaging:tag: None 2024-09-16T00:21:12.972 INFO:teuthology.packaging:branch: squid 2024-09-16T00:21:12.972 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:21:12.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-16T00:21:13.179 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-16T00:21:13.182 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T00:21:13.187 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T00:21:13.187 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T00:21:13.194 INFO:teuthology.task.internal:Saving configuration 2024-09-16T00:21:13.213 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T00:21:13.219 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T00:21:13.246 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:14:20.000454', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHCbibadsvHOL/gj/VPqzsA4NHKTi00ueduN8gAssw/kKP/z/F/jWYchlyqXt1bQsMQmmpfUfgTIFbMUCp+Z1HA='} 2024-09-16T00:21:13.270 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:14:19.997934', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLg2Snc1G87T0ztM9lLZ9F56LLPw0xGGKStWI1voQ8UnmYK9CPNgD5uapE7WSyDFiIRQ5ctQZK5Ng/yIC6pkbXo='} 2024-09-16T00:21:13.294 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:14:19.999639', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWfzkCbCKpSe1tlGzDnfAcLHYsVvEpZUAEkwAI9ZjQ0xJreDUUl9TcxylU2GjM+P3z28PZMLblr+Vkhrg3IbwE='} 2024-09-16T00:21:13.294 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T00:21:13.315 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-16T00:21:13.315 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-16T00:21:13.315 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-16T00:21:13.315 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T00:21:13.404 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f542e06fbe0>, signals=[15]) 2024-09-16T00:21:13.404 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T00:21:13.419 INFO:teuthology.task.internal:Opening connections... 2024-09-16T00:21:13.419 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:21:13.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:21:13.501 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-09-16T00:21:13.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:21:13.599 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T00:21:13.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:21:13.679 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T00:21:13.690 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-16T00:21:13.709 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-16T00:21:13.709 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-16T00:21:13.767 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:21:13.768 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:21:13.769 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:21:13.769 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-16T00:21:13.793 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-09-16T00:21:13.812 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-09-16T00:21:13.812 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:21:13.871 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:21:13.872 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:21:13.872 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:21:13.872 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:21:13.872 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-09-16T00:21:13.893 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-09-16T00:21:13.912 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-09-16T00:21:13.912 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:21:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:21:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:21:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:21:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:21:13.971 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-09-16T00:21:13.992 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T00:21:14.028 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T00:21:14.032 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T00:21:14.033 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:21:14.035 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:21:14.038 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:21:14.055 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T00:21:14.059 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T00:21:14.060 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:21:14.094 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:21:14.097 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:21:14.208 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T00:21:14.285 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T00:21:14.303 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-16T00:21:14.304 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-16T00:21:14.304 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-16T00:21:14.304 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:21:14.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:21:14.323 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-16T00:21:14.381 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-503.el9.x86_64 2024-09-16T00:21:14.382 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-503.el9.x86_64 2024-09-16T00:21:14.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-09-16T00:21:16.592 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-09-16T00:21:21.176 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 20 MB 00:03 2024-09-16T00:21:27.060 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-16T00:21:28.610 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-16T00:21:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-16T00:21:34.969 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T00:21:37.053 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T00:21:37.053 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T00:21:37.135 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T00:21:37.159 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T00:21:37.159 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T00:21:37.160 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-16T00:21:37.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:21:37.161 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 79 M 2024-09-16T00:21:37.161 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 126 M 2024-09-16T00:21:37.161 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T00:21:37.273 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 248 kB/s | 23 kB 00:00 2024-09-16T00:21:40.098 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.4 MB/s | 16 MB 00:02 2024-09-16T00:21:41.456 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.8 MB/s | 29 MB 00:04 2024-09-16T00:21:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.7 MB/s | 35 MB 00:05 2024-09-16T00:21:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:21:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:Total 15 MB/s | 79 MB 00:05 2024-09-16T00:21:43.036 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T00:21:43.430 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T00:21:43.430 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T00:21:44.420 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T00:21:44.420 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T00:21:47.001 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T00:21:47.684 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:21:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:21:50.571 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:21:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:21:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:21:55.869 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:22:01.014 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:22:40.341 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:22:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:22:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:22:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:22:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:22:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:22:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:22:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:22:41.518 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T00:22:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T00:22:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T00:22:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T00:22:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T00:22:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:22:41.519 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T00:22:41.850 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:22:41.942 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:22:41.942 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:22:41.943 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T00:22:41.943 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T00:22:41.943 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T00:22:41.943 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-16T00:22:41.943 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T00:22:41.943 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:22:42.001 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:22:42.002 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:22:42.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-09-16T00:22:42.033 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-09-16T00:22:42.033 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-09-16T00:22:42.033 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-09-16T00:22:42.033 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 82.el9 2024-09-16T00:22:42.033 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8274670 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-09-16T00:22:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:22:42.035 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T00:22:42.035 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T00:22:42.035 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T00:22:42.035 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-09-16T00:22:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:22:42.035 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T00:22:42.036 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T00:22:42.036 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-505.el9.x86_64 2024-09-16T00:22:42.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T00:22:42.719 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-16T00:22:44.045 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T00:22:44.082 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-16T00:22:44.084 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-16T00:22:44.099 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.z4R99n5lfv 2024-09-16T00:22:44.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.z4R99n5lfv 2024-09-16T00:22:44.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.z4R99n5lfv 2024-09-16T00:22:44.316 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.z4R99n5lfv is 6KB 2024-09-16T00:22:44.365 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.z4R99n5lfv 2024-09-16T00:22:44.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T00:22:44.442 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T00:22:44.442 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T00:22:44.442 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T00:22:44.442 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T00:22:44.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T00:22:44.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-16T00:23:14.590 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-16T00:23:14.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T00:23:14.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:23:32.925 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-16T00:23:41.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T00:23:41.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:24:00.324 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-16T00:24:12.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T00:24:12.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:24:14.313 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-16T00:24:14.738 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T00:24:14.739 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-16T00:24:14.739 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-16T00:24:14.754 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-505.el9.x86_64 2024-09-16T00:24:14.754 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T00:24:14.754 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T00:24:14.754 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T00:24:15.756 DEBUG:teuthology.parallel:result is None 2024-09-16T00:24:15.756 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T00:24:15.764 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T00:24:15.764 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:24:15.767 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:24:15.770 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:24:15.788 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T00:24:15.795 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T00:24:15.805 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T00:24:15.805 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:24:15.824 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:24:15.829 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:24:15.931 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T00:24:15.940 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T00:24:15.940 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:24:15.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:24:15.957 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:24:15.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:24:15.975 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:24:15.977 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:24:16.000 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:24:16.029 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:24:16.049 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:24:16.060 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:24:16.072 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:24:16.079 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:24:16.122 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:24:16.124 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T00:24:16.132 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T00:24:16.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:24:16.134 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:24:16.137 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:24:16.167 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T00:24:16.176 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T00:24:16.176 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:24:16.201 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:24:16.205 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:24:16.225 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:24:16.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:24:16.346 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:24:16.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:24:16.471 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:24:16.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:24:16.575 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:24:16.621 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:24:16.675 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:24:16.761 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:24:16.804 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:24:16.804 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:24:16.872 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:24:16.915 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:24:17.007 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:24:17.053 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:24:17.096 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:24:17.097 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:24:17.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-16T00:24:17.168 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-09-16T00:24:17.170 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-09-16T00:24:17.206 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:24:17.213 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:24:17.242 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:24:17.608 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T00:24:17.615 INFO:teuthology.task.internal:Starting timer... 2024-09-16T00:24:17.615 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T00:24:17.630 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T00:24:17.639 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-16T00:24:17.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-09-16T00:24:17.688 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-09-16T00:24:17.690 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-09-16T00:24:17.739 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-09-16T00:24:17.741 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-09-16T00:24:17.806 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-09-16T00:24:17.808 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T00:24:17.809 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-09-16T00:24:17.832 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-09-16T00:24:17.832 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-09-16T00:24:17.860 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-09-16T00:24:17.861 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-09-16T00:24:17.882 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-09-16T00:24:17.882 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive'} 2024-09-16T00:24:17.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:24:17.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:24:17.910 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:24:17.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:24:17.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:24:17.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:24:17.972 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T00:24:17.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:24:18.020 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:24:18.051 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:24:18.098 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T00:24:18.109 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T00:24:18.112 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T00:24:18.227 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T00:24:18.238 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T00:24:18.239 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi081.front.sepia.ceph.com,smithi102.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T00:28:18.573 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com')] 2024-09-16T00:28:18.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T00:28:18.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:28:18.655 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-16T00:28:18.736 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T00:28:18.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-16T00:28:18.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:28:18.810 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-09-16T00:28:18.849 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-16T00:28:18.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-16T00:28:18.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:28:18.925 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-16T00:28:19.008 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-16T00:28:19.009 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T00:28:19.020 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T00:28:19.021 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:28:19.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:28:19.023 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:28:19.024 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:28:19.026 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:28:19.026 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:28:19.052 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:28:19.052 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:28:19.067 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:28:19.067 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:28:19.078 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:28:19.091 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-09-16T00:28:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-09-16T00:28:19.093 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:28:19.102 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-09-16T00:28:19.104 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-09-16T00:28:19.116 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:28:19.117 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-09-16T00:28:19.119 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:28:19.128 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-09-16T00:28:19.130 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:28:19.134 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:28:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:28:19.157 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:28:19.190 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:28:19.193 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:28:19.193 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-09-16T00:28:19.193 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.193 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.194 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.194 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.194 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:28:19.197 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:28:19.197 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-16T00:28:19.197 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.197 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.197 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.197 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.212 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:28:19.215 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:28:19.215 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-16T00:28:19.215 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.215 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.215 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.215 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:28:19.216 INFO:teuthology.run_tasks:Running task install... 2024-09-16T00:28:19.224 DEBUG:teuthology.task.install:project ceph 2024-09-16T00:28:19.224 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T00:28:19.225 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T00:28:19.225 INFO:teuthology.task.install:Using flavor: default 2024-09-16T00:28:19.233 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T00:28:19.233 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T00:28:19.234 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:28:19.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:28:19.236 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:28:19.237 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:28:19.237 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:28:19.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:28:19.389 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:28:19.390 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:28:19.393 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:28:19.393 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:28:19.397 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:28:19.398 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:28:19.498 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:28:19.498 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:28:19.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:28:19.510 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:28:19.510 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:28:19.510 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:28:19.514 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:28:19.514 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:28:19.514 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:28:19.526 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:28:19.527 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:28:19.537 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:28:19.537 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:28:19.541 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:28:19.541 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:28:19.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:28:19.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:28:19.607 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:28:19.672 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:28:19.678 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:28:19.682 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:28:19.739 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-09-16T00:28:19.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-09-16T00:28:19.742 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2024-09-16T00:28:19.743 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2024-09-16T00:28:19.746 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-09-16T00:28:19.747 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-09-16T00:28:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:45 files removed 2024-09-16T00:28:19.973 INFO:teuthology.orchestra.run.smithi102.stdout:45 files removed 2024-09-16T00:28:19.975 INFO:teuthology.orchestra.run.smithi031.stdout:45 files removed 2024-09-16T00:28:19.994 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:28:20.000 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:28:20.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:28:20.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 316 kB/s | 83 kB 00:00 2024-09-16T00:28:20.545 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 304 kB/s | 83 kB 00:00 2024-09-16T00:28:20.570 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 312 kB/s | 83 kB 00:00 2024-09-16T00:28:20.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-16T00:28:21.067 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-16T00:28:21.067 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-16T00:28:21.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 8.8 kB/s | 1.9 kB 00:00 2024-09-16T00:28:21.104 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T00:28:21.130 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T00:28:21.680 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-16T00:28:22.171 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.2 MB 00:01 2024-09-16T00:28:23.338 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-16T00:28:23.504 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.2 MB 00:02 2024-09-16T00:28:25.280 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-16T00:28:25.589 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-16T00:28:27.697 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-16T00:28:29.231 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 19 kB 00:00 2024-09-16T00:28:29.840 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-16T00:28:30.010 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-16T00:28:30.480 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.5 MB 00:01 2024-09-16T00:28:31.207 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 122 kB/s | 19 kB 00:00 2024-09-16T00:28:32.112 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-16T00:28:32.206 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-16T00:28:33.102 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T00:28:35.425 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T00:28:37.543 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-16T00:28:37.685 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:28:37.686 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:28:37.697 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:28:37.698 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:28:37.699 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:28:37.700 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:28:37.701 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:28:37.703 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:28:37.704 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:28:37.767 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T00:28:37.780 INFO:teuthology.orchestra.run.smithi102.stdout:====================================================================================== 2024-09-16T00:28:37.780 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T00:28:37.780 INFO:teuthology.orchestra.run.smithi102.stdout:====================================================================================== 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:28:37.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:28:37.782 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:28:37.783 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:28:37.784 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:28:37.785 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:28:37.786 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:28:37.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:28:37.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:28:37.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:28:37.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:28:37.791 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:28:37.792 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:28:37.792 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:28:37.792 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:28:37.792 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:28:37.792 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:28:37.792 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:28:37.792 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:28:37.792 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:28:37.792 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:28:37.793 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:28:37.794 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T00:28:37.794 INFO:teuthology.orchestra.run.smithi102.stdout:====================================================================================== 2024-09-16T00:28:37.794 INFO:teuthology.orchestra.run.smithi102.stdout:Install 148 Packages 2024-09-16T00:28:37.794 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 8 Packages 2024-09-16T00:28:37.794 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:28:37.795 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 226 M 2024-09-16T00:28:37.795 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T00:28:38.660 INFO:teuthology.orchestra.run.smithi102.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-16T00:28:38.750 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-16T00:28:38.835 INFO:teuthology.orchestra.run.smithi102.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 5.0 MB/s | 896 kB 00:00 2024-09-16T00:28:38.927 INFO:teuthology.orchestra.run.smithi102.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.5 MB/s | 145 kB 00:00 2024-09-16T00:28:39.044 INFO:teuthology.orchestra.run.smithi102.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-16T00:28:39.136 INFO:teuthology.orchestra.run.smithi102.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-16T00:28:39.186 INFO:teuthology.orchestra.run.smithi102.stdout:(6/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 9.2 MB/s | 2.4 MB 00:00 2024-09-16T00:28:39.437 INFO:teuthology.orchestra.run.smithi102.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-16T00:28:39.704 INFO:teuthology.orchestra.run.smithi102.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 18 MB/s | 21 MB 00:01 2024-09-16T00:28:39.745 INFO:teuthology.orchestra.run.smithi102.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 612 kB/s | 25 kB 00:00 2024-09-16T00:28:39.829 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:28:39.829 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:28:39.841 INFO:teuthology.orchestra.run.smithi081.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:28:39.842 INFO:teuthology.orchestra.run.smithi081.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:28:39.843 INFO:teuthology.orchestra.run.smithi081.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:28:39.843 INFO:teuthology.orchestra.run.smithi081.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:28:39.845 INFO:teuthology.orchestra.run.smithi081.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:28:39.847 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:28:39.848 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:28:39.912 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-16T00:28:39.922 INFO:teuthology.orchestra.run.smithi081.stdout:====================================================================================== 2024-09-16T00:28:39.922 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-16T00:28:39.922 INFO:teuthology.orchestra.run.smithi081.stdout:====================================================================================== 2024-09-16T00:28:39.922 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:28:39.923 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:28:39.924 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-09-16T00:28:39.925 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:28:39.926 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:28:39.927 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:28:39.928 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:28:39.929 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:28:39.930 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:28:39.931 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:28:39.932 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:28:39.933 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:28:39.934 INFO:teuthology.orchestra.run.smithi081.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout:====================================================================================== 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout:Install 148 Packages 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 8 Packages 2024-09-16T00:28:39.935 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:28:39.936 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 226 M 2024-09-16T00:28:39.936 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-16T00:28:40.129 INFO:teuthology.orchestra.run.smithi102.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 18 MB/s | 17 MB 00:00 2024-09-16T00:28:40.254 INFO:teuthology.orchestra.run.smithi102.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 13 MB/s | 11 MB 00:00 2024-09-16T00:28:40.279 INFO:teuthology.orchestra.run.smithi102.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.5 MB/s | 228 kB 00:00 2024-09-16T00:28:40.305 INFO:teuthology.orchestra.run.smithi102.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 542 kB/s | 27 kB 00:00 2024-09-16T00:28:40.330 INFO:teuthology.orchestra.run.smithi102.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 631 kB/s | 32 kB 00:00 2024-09-16T00:28:40.372 INFO:teuthology.orchestra.run.smithi102.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 751 kB 00:00 2024-09-16T00:28:40.397 INFO:teuthology.orchestra.run.smithi102.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.4 MB/s | 163 kB 00:00 2024-09-16T00:28:40.423 INFO:teuthology.orchestra.run.smithi102.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-16T00:28:40.477 INFO:teuthology.orchestra.run.smithi081.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-16T00:28:40.481 INFO:teuthology.orchestra.run.smithi102.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-16T00:28:40.532 INFO:teuthology.orchestra.run.smithi102.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 900 kB/s | 45 kB 00:00 2024-09-16T00:28:40.598 INFO:teuthology.orchestra.run.smithi102.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.0 MB/s | 136 kB 00:00 2024-09-16T00:28:40.666 INFO:teuthology.orchestra.run.smithi102.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.4 MB/s | 161 kB 00:00 2024-09-16T00:28:40.783 INFO:teuthology.orchestra.run.smithi102.stdout:(22/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-16T00:28:40.816 INFO:teuthology.orchestra.run.smithi102.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.1 MB/s | 320 kB 00:00 2024-09-16T00:28:40.842 INFO:teuthology.orchestra.run.smithi102.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-16T00:28:40.844 INFO:teuthology.orchestra.run.smithi081.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 2.4 MB/s | 896 kB 00:00 2024-09-16T00:28:40.867 INFO:teuthology.orchestra.run.smithi102.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T00:28:40.894 INFO:teuthology.orchestra.run.smithi102.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.6 MB/s | 85 kB 00:00 2024-09-16T00:28:40.920 INFO:teuthology.orchestra.run.smithi081.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.9 MB/s | 145 kB 00:00 2024-09-16T00:28:40.969 INFO:teuthology.orchestra.run.smithi102.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.2 MB/s | 170 kB 00:00 2024-09-16T00:28:41.019 INFO:teuthology.orchestra.run.smithi102.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 20 MB/s | 3.1 MB 00:00 2024-09-16T00:28:41.036 INFO:teuthology.orchestra.run.smithi102.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 476 kB/s | 32 kB 00:00 2024-09-16T00:28:41.070 INFO:teuthology.orchestra.run.smithi102.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.8 MB/s | 146 kB 00:00 2024-09-16T00:28:41.177 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:28:41.178 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:28:41.189 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:28:41.190 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:28:41.191 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:28:41.192 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:28:41.194 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:28:41.196 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:28:41.197 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:28:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T00:28:41.283 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-16T00:28:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:28:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:28:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:28:41.286 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:28:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:28:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:28:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:28:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:28:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:28:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:28:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:28:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:28:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout:Install 148 Packages 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 8 Packages 2024-09-16T00:28:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:28:41.297 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 226 M 2024-09-16T00:28:41.297 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T00:28:41.379 INFO:teuthology.orchestra.run.smithi102.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 10 MB/s | 3.6 MB 00:00 2024-09-16T00:28:41.512 INFO:teuthology.orchestra.run.smithi102.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-16T00:28:41.546 INFO:teuthology.orchestra.run.smithi102.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 1.5 MB/s | 249 kB 00:00 2024-09-16T00:28:41.553 INFO:teuthology.orchestra.run.smithi081.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 4.5 MB/s | 5.4 MB 00:01 2024-09-16T00:28:41.563 INFO:teuthology.orchestra.run.smithi102.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 982 kB/s | 49 kB 00:00 2024-09-16T00:28:41.588 INFO:teuthology.orchestra.run.smithi102.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 394 kB/s | 16 kB 00:00 2024-09-16T00:28:41.621 INFO:teuthology.orchestra.run.smithi102.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.8 MB/s | 285 kB 00:00 2024-09-16T00:28:41.655 INFO:teuthology.orchestra.run.smithi102.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 11 MB/s | 765 kB 00:00 2024-09-16T00:28:41.680 INFO:teuthology.orchestra.run.smithi102.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 311 kB/s | 18 kB 00:00 2024-09-16T00:28:41.745 INFO:teuthology.orchestra.run.smithi081.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 2.9 MB/s | 2.4 MB 00:00 2024-09-16T00:28:41.789 INFO:teuthology.orchestra.run.smithi102.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-16T00:28:41.804 INFO:teuthology.orchestra.run.smithi081.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 6.1 MB/s | 1.5 MB 00:00 2024-09-16T00:28:41.856 INFO:teuthology.orchestra.run.smithi102.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-16T00:28:41.881 INFO:teuthology.orchestra.run.smithi102.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-16T00:28:41.907 INFO:teuthology.orchestra.run.smithi102.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-16T00:28:41.965 INFO:teuthology.orchestra.run.smithi102.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-16T00:28:41.990 INFO:teuthology.orchestra.run.smithi102.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-16T00:28:42.024 INFO:teuthology.orchestra.run.smithi102.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-16T00:28:42.050 INFO:teuthology.orchestra.run.smithi102.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-16T00:28:42.083 INFO:teuthology.orchestra.run.smithi102.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-16T00:28:42.109 INFO:teuthology.orchestra.run.smithi031.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-16T00:28:42.301 INFO:teuthology.orchestra.run.smithi031.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 4.6 MB/s | 896 kB 00:00 2024-09-16T00:28:42.376 INFO:teuthology.orchestra.run.smithi031.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.9 MB/s | 145 kB 00:00 2024-09-16T00:28:42.510 INFO:teuthology.orchestra.run.smithi031.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-16T00:28:42.542 INFO:teuthology.orchestra.run.smithi102.stdout:(48/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 17 MB/s | 49 MB 00:02 2024-09-16T00:28:42.635 INFO:teuthology.orchestra.run.smithi031.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 9.2 MB/s | 2.4 MB 00:00 2024-09-16T00:28:42.676 INFO:teuthology.orchestra.run.smithi102.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 803 kB/s | 107 kB 00:00 2024-09-16T00:28:42.677 INFO:teuthology.orchestra.run.smithi031.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-16T00:28:42.718 INFO:teuthology.orchestra.run.smithi102.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-09-16T00:28:42.752 INFO:teuthology.orchestra.run.smithi102.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.4 MB/s | 943 kB 00:00 2024-09-16T00:28:42.785 INFO:teuthology.orchestra.run.smithi102.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 986 kB/s | 33 kB 00:00 2024-09-16T00:28:42.811 INFO:teuthology.orchestra.run.smithi102.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-16T00:28:42.812 INFO:teuthology.orchestra.run.smithi081.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 4.4 MB/s | 4.7 MB 00:01 2024-09-16T00:28:42.844 INFO:teuthology.orchestra.run.smithi102.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 550 kB/s | 18 kB 00:00 2024-09-16T00:28:42.878 INFO:teuthology.orchestra.run.smithi102.stdout:(55/156): automake-1.16.2-8.el9.noarch.rpm 832 kB/s | 689 kB 00:00 2024-09-16T00:28:42.895 INFO:teuthology.orchestra.run.smithi102.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-16T00:28:42.953 INFO:teuthology.orchestra.run.smithi102.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-09-16T00:28:42.970 INFO:teuthology.orchestra.run.smithi102.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-09-16T00:28:42.995 INFO:teuthology.orchestra.run.smithi102.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-16T00:28:43.029 INFO:teuthology.orchestra.run.smithi102.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-16T00:28:43.046 INFO:teuthology.orchestra.run.smithi102.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 489 kB/s | 24 kB 00:00 2024-09-16T00:28:43.079 INFO:teuthology.orchestra.run.smithi102.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-16T00:28:43.096 INFO:teuthology.orchestra.run.smithi102.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-09-16T00:28:43.130 INFO:teuthology.orchestra.run.smithi102.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T00:28:43.155 INFO:teuthology.orchestra.run.smithi102.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-16T00:28:43.160 INFO:teuthology.orchestra.run.smithi031.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 9.0 MB/s | 4.7 MB 00:00 2024-09-16T00:28:43.189 INFO:teuthology.orchestra.run.smithi102.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-16T00:28:43.273 INFO:teuthology.orchestra.run.smithi102.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-16T00:28:43.348 INFO:teuthology.orchestra.run.smithi102.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 23 MB/s | 4.9 MB 00:00 2024-09-16T00:28:43.382 INFO:teuthology.orchestra.run.smithi102.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-16T00:28:43.407 INFO:teuthology.orchestra.run.smithi102.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-16T00:28:43.465 INFO:teuthology.orchestra.run.smithi102.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-16T00:28:43.541 INFO:teuthology.orchestra.run.smithi102.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-09-16T00:28:43.566 INFO:teuthology.orchestra.run.smithi102.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-09-16T00:28:43.608 INFO:teuthology.orchestra.run.smithi102.stdout:(74/156): postgresql-13.16-1.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-09-16T00:28:43.633 INFO:teuthology.orchestra.run.smithi102.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 915 kB/s | 61 kB 00:00 2024-09-16T00:28:43.636 INFO:teuthology.orchestra.run.smithi031.stdout:(8/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86_ 18 MB/s | 17 MB 00:00 2024-09-16T00:28:43.692 INFO:teuthology.orchestra.run.smithi102.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-16T00:28:43.726 INFO:teuthology.orchestra.run.smithi102.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-16T00:28:43.751 INFO:teuthology.orchestra.run.smithi102.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-16T00:28:43.776 INFO:teuthology.orchestra.run.smithi102.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-16T00:28:43.788 INFO:teuthology.orchestra.run.smithi081.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 6.2 MB/s | 21 MB 00:03 2024-09-16T00:28:43.801 INFO:teuthology.orchestra.run.smithi102.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-16T00:28:43.827 INFO:teuthology.orchestra.run.smithi102.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-16T00:28:43.838 INFO:teuthology.orchestra.run.smithi081.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 507 kB/s | 25 kB 00:00 2024-09-16T00:28:43.844 INFO:teuthology.orchestra.run.smithi031.stdout:(9/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 21 MB 00:01 2024-09-16T00:28:43.869 INFO:teuthology.orchestra.run.smithi031.stdout:(10/156): ceph-selinux-19.1.1-326.g8921c8ef.el9 108 kB/s | 25 kB 00:00 2024-09-16T00:28:43.911 INFO:teuthology.orchestra.run.smithi102.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-16T00:28:43.928 INFO:teuthology.orchestra.run.smithi031.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 3.8 MB/s | 228 kB 00:00 2024-09-16T00:28:43.936 INFO:teuthology.orchestra.run.smithi102.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-16T00:28:43.970 INFO:teuthology.orchestra.run.smithi102.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-16T00:28:43.995 INFO:teuthology.orchestra.run.smithi102.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-16T00:28:43.995 INFO:teuthology.orchestra.run.smithi031.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 407 kB/s | 27 kB 00:00 2024-09-16T00:28:44.012 INFO:teuthology.orchestra.run.smithi102.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-16T00:28:44.062 INFO:teuthology.orchestra.run.smithi031.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 475 kB/s | 32 kB 00:00 2024-09-16T00:28:44.087 INFO:teuthology.orchestra.run.smithi102.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-09-16T00:28:44.121 INFO:teuthology.orchestra.run.smithi102.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-16T00:28:44.146 INFO:teuthology.orchestra.run.smithi102.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-09-16T00:28:44.163 INFO:teuthology.orchestra.run.smithi102.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-16T00:28:44.180 INFO:teuthology.orchestra.run.smithi031.stdout:(14/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 10 MB/s | 11 MB 00:01 2024-09-16T00:28:44.188 INFO:teuthology.orchestra.run.smithi102.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-16T00:28:44.205 INFO:teuthology.orchestra.run.smithi102.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 913 kB/s | 54 kB 00:00 2024-09-16T00:28:44.212 INFO:teuthology.orchestra.run.smithi031.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 4.9 MB/s | 751 kB 00:00 2024-09-16T00:28:44.222 INFO:teuthology.orchestra.run.smithi102.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-16T00:28:44.237 INFO:teuthology.orchestra.run.smithi031.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.7 MB/s | 163 kB 00:00 2024-09-16T00:28:44.246 INFO:teuthology.orchestra.run.smithi081.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 6.9 MB/s | 17 MB 00:02 2024-09-16T00:28:44.247 INFO:teuthology.orchestra.run.smithi102.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-16T00:28:44.263 INFO:teuthology.orchestra.run.smithi031.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-16T00:28:44.281 INFO:teuthology.orchestra.run.smithi102.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-16T00:28:44.297 INFO:teuthology.orchestra.run.smithi081.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.5 MB/s | 228 kB 00:00 2024-09-16T00:28:44.297 INFO:teuthology.orchestra.run.smithi102.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-16T00:28:44.305 INFO:teuthology.orchestra.run.smithi031.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 7.4 MB/s | 506 kB 00:00 2024-09-16T00:28:44.315 INFO:teuthology.orchestra.run.smithi102.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-16T00:28:44.338 INFO:teuthology.orchestra.run.smithi081.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 649 kB/s | 27 kB 00:00 2024-09-16T00:28:44.357 INFO:teuthology.orchestra.run.smithi102.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-16T00:28:44.372 INFO:teuthology.orchestra.run.smithi031.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 677 kB/s | 45 kB 00:00 2024-09-16T00:28:44.399 INFO:teuthology.orchestra.run.smithi102.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.9 MB/s | 304 kB 00:00 2024-09-16T00:28:44.414 INFO:teuthology.orchestra.run.smithi081.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 423 kB/s | 32 kB 00:00 2024-09-16T00:28:44.424 INFO:teuthology.orchestra.run.smithi102.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-16T00:28:44.449 INFO:teuthology.orchestra.run.smithi102.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T00:28:44.455 INFO:teuthology.orchestra.run.smithi031.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-16T00:28:44.474 INFO:teuthology.orchestra.run.smithi102.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-09-16T00:28:44.531 INFO:teuthology.orchestra.run.smithi031.stdout:(21/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 20 MB/s | 5.3 MB 00:00 2024-09-16T00:28:44.547 INFO:teuthology.orchestra.run.smithi081.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 5.5 MB/s | 751 kB 00:00 2024-09-16T00:28:44.556 INFO:teuthology.orchestra.run.smithi031.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.6 MB/s | 161 kB 00:00 2024-09-16T00:28:44.589 INFO:teuthology.orchestra.run.smithi031.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 5.4 MB/s | 320 kB 00:00 2024-09-16T00:28:44.598 INFO:teuthology.orchestra.run.smithi081.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 3.2 MB/s | 163 kB 00:00 2024-09-16T00:28:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-16T00:28:44.648 INFO:teuthology.orchestra.run.smithi081.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-16T00:28:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-16T00:28:44.650 INFO:teuthology.orchestra.run.smithi102.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-16T00:28:44.674 INFO:teuthology.orchestra.run.smithi031.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-16T00:28:44.700 INFO:teuthology.orchestra.run.smithi102.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-16T00:28:44.732 INFO:teuthology.orchestra.run.smithi081.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-16T00:28:44.734 INFO:teuthology.orchestra.run.smithi102.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-16T00:28:44.749 INFO:teuthology.orchestra.run.smithi031.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.2 MB/s | 170 kB 00:00 2024-09-16T00:28:44.751 INFO:teuthology.orchestra.run.smithi102.stdout:(106/156): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2024-09-16T00:28:44.768 INFO:teuthology.orchestra.run.smithi102.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 577 kB/s | 19 kB 00:00 2024-09-16T00:28:44.793 INFO:teuthology.orchestra.run.smithi102.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.4 MB/s | 26 kB 00:00 2024-09-16T00:28:44.818 INFO:teuthology.orchestra.run.smithi102.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-16T00:28:44.824 INFO:teuthology.orchestra.run.smithi031.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 18 MB/s | 3.1 MB 00:00 2024-09-16T00:28:44.844 INFO:teuthology.orchestra.run.smithi102.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-16T00:28:44.850 INFO:teuthology.orchestra.run.smithi031.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 319 kB/s | 32 kB 00:00 2024-09-16T00:28:44.869 INFO:teuthology.orchestra.run.smithi102.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-16T00:28:44.911 INFO:teuthology.orchestra.run.smithi031.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.7 MB/s | 146 kB 00:00 2024-09-16T00:28:44.911 INFO:teuthology.orchestra.run.smithi102.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-16T00:28:44.961 INFO:teuthology.orchestra.run.smithi102.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-16T00:28:44.986 INFO:teuthology.orchestra.run.smithi102.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:28:45.012 INFO:teuthology.orchestra.run.smithi102.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T00:28:45.037 INFO:teuthology.orchestra.run.smithi102.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-16T00:28:45.062 INFO:teuthology.orchestra.run.smithi102.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-16T00:28:45.104 INFO:teuthology.orchestra.run.smithi102.stdout:(118/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-09-16T00:28:45.138 INFO:teuthology.orchestra.run.smithi102.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-16T00:28:45.171 INFO:teuthology.orchestra.run.smithi102.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.9 MB/s | 220 kB 00:00 2024-09-16T00:28:45.208 INFO:teuthology.orchestra.run.smithi031.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 9.9 MB/s | 3.6 MB 00:00 2024-09-16T00:28:45.238 INFO:teuthology.orchestra.run.smithi102.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-09-16T00:28:45.272 INFO:teuthology.orchestra.run.smithi102.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-09-16T00:28:45.297 INFO:teuthology.orchestra.run.smithi102.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-16T00:28:45.323 INFO:teuthology.orchestra.run.smithi102.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-16T00:28:45.334 INFO:teuthology.orchestra.run.smithi031.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 18 MB/s | 7.4 MB 00:00 2024-09-16T00:28:45.348 INFO:teuthology.orchestra.run.smithi102.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-09-16T00:28:45.359 INFO:teuthology.orchestra.run.smithi031.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 1.6 MB/s | 249 kB 00:00 2024-09-16T00:28:45.373 INFO:teuthology.orchestra.run.smithi102.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-16T00:28:45.384 INFO:teuthology.orchestra.run.smithi031.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 990 kB/s | 49 kB 00:00 2024-09-16T00:28:45.398 INFO:teuthology.orchestra.run.smithi102.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-16T00:28:45.409 INFO:teuthology.orchestra.run.smithi031.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 327 kB/s | 16 kB 00:00 2024-09-16T00:28:45.424 INFO:teuthology.orchestra.run.smithi102.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-16T00:28:45.443 INFO:teuthology.orchestra.run.smithi031.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.7 MB/s | 285 kB 00:00 2024-09-16T00:28:45.449 INFO:teuthology.orchestra.run.smithi102.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:28:45.474 INFO:teuthology.orchestra.run.smithi102.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T00:28:45.493 INFO:teuthology.orchestra.run.smithi031.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 8.9 MB/s | 765 kB 00:00 2024-09-16T00:28:45.499 INFO:teuthology.orchestra.run.smithi102.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-16T00:28:45.519 INFO:teuthology.orchestra.run.smithi031.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 243 kB/s | 18 kB 00:00 2024-09-16T00:28:45.533 INFO:teuthology.orchestra.run.smithi102.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-16T00:28:45.540 INFO:teuthology.orchestra.run.smithi081.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 6.6 MB/s | 5.3 MB 00:00 2024-09-16T00:28:45.558 INFO:teuthology.orchestra.run.smithi102.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-16T00:28:45.592 INFO:teuthology.orchestra.run.smithi102.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-16T00:28:45.599 INFO:teuthology.orchestra.run.smithi081.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 775 kB/s | 45 kB 00:00 2024-09-16T00:28:45.617 INFO:teuthology.orchestra.run.smithi102.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-16T00:28:45.643 INFO:teuthology.orchestra.run.smithi102.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-16T00:28:45.649 INFO:teuthology.orchestra.run.smithi081.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.7 MB/s | 136 kB 00:00 2024-09-16T00:28:45.668 INFO:teuthology.orchestra.run.smithi102.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-16T00:28:45.694 INFO:teuthology.orchestra.run.smithi102.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:28:45.708 INFO:teuthology.orchestra.run.smithi081.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.7 MB/s | 161 kB 00:00 2024-09-16T00:28:45.719 INFO:teuthology.orchestra.run.smithi102.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-16T00:28:45.744 INFO:teuthology.orchestra.run.smithi102.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-16T00:28:45.769 INFO:teuthology.orchestra.run.smithi102.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-16T00:28:45.792 INFO:teuthology.orchestra.run.smithi081.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-16T00:28:45.795 INFO:teuthology.orchestra.run.smithi102.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T00:28:45.820 INFO:teuthology.orchestra.run.smithi102.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T00:28:45.842 INFO:teuthology.orchestra.run.smithi081.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.9 MB/s | 300 kB 00:00 2024-09-16T00:28:45.870 INFO:teuthology.orchestra.run.smithi102.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-09-16T00:28:45.888 INFO:teuthology.orchestra.run.smithi102.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-09-16T00:28:45.892 INFO:teuthology.orchestra.run.smithi081.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T00:28:45.913 INFO:teuthology.orchestra.run.smithi102.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-16T00:28:45.938 INFO:teuthology.orchestra.run.smithi102.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-16T00:28:45.943 INFO:teuthology.orchestra.run.smithi081.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T00:28:45.988 INFO:teuthology.orchestra.run.smithi102.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T00:28:46.076 INFO:teuthology.orchestra.run.smithi081.stdout:(26/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 3.2 MB/s | 11 MB 00:03 2024-09-16T00:28:46.177 INFO:teuthology.orchestra.run.smithi031.stdout:(39/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 21 MB/s | 49 MB 00:02 2024-09-16T00:28:46.180 INFO:teuthology.orchestra.run.smithi102.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-09-16T00:28:46.185 INFO:teuthology.orchestra.run.smithi081.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.5 MB/s | 170 kB 00:00 2024-09-16T00:28:46.203 INFO:teuthology.orchestra.run.smithi031.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 113 kB/s | 80 kB 00:00 2024-09-16T00:28:46.227 INFO:teuthology.orchestra.run.smithi081.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 768 kB/s | 32 kB 00:00 2024-09-16T00:28:46.247 INFO:teuthology.orchestra.run.smithi102.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-16T00:28:46.277 INFO:teuthology.orchestra.run.smithi081.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-16T00:28:46.298 INFO:teuthology.orchestra.run.smithi102.stdout:(151/156): librados2-19.1.1-326.g8921c8ef.el9.x 8.9 MB/s | 3.4 MB 00:00 2024-09-16T00:28:46.323 INFO:teuthology.orchestra.run.smithi102.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 542 kB/s | 41 kB 00:00 2024-09-16T00:28:46.373 INFO:teuthology.orchestra.run.smithi102.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 9.4 MB/s | 1.8 MB 00:00 2024-09-16T00:28:46.377 INFO:teuthology.orchestra.run.smithi081.stdout:(30/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 7.1 MB/s | 3.1 MB 00:00 2024-09-16T00:28:46.448 INFO:teuthology.orchestra.run.smithi102.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-09-16T00:28:46.482 INFO:teuthology.orchestra.run.smithi102.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.3 MB/s | 249 kB 00:00 2024-09-16T00:28:46.532 INFO:teuthology.orchestra.run.smithi102.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 6.5 MB/s | 1.4 MB 00:00 2024-09-16T00:28:46.538 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:28:46.538 INFO:teuthology.orchestra.run.smithi102.stdout:Total 26 MB/s | 226 MB 00:08 2024-09-16T00:28:46.578 INFO:teuthology.orchestra.run.smithi031.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 675 kB/s | 253 kB 00:00 2024-09-16T00:28:46.653 INFO:teuthology.orchestra.run.smithi031.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 707 kB/s | 801 kB 00:01 2024-09-16T00:28:46.720 INFO:teuthology.orchestra.run.smithi031.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 352 kB/s | 191 kB 00:00 2024-09-16T00:28:46.762 INFO:teuthology.orchestra.run.smithi031.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 106 kB 00:00 2024-09-16T00:28:46.820 INFO:teuthology.orchestra.run.smithi031.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-16T00:28:46.828 INFO:teuthology.orchestra.run.smithi081.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 6.5 MB/s | 3.6 MB 00:00 2024-09-16T00:28:46.871 INFO:teuthology.orchestra.run.smithi031.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-16T00:28:46.895 INFO:teuthology.orchestra.run.smithi081.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 3.7 MB/s | 249 kB 00:00 2024-09-16T00:28:46.937 INFO:teuthology.orchestra.run.smithi081.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 1.2 MB/s | 49 kB 00:00 2024-09-16T00:28:47.021 INFO:teuthology.orchestra.run.smithi081.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-16T00:28:47.021 INFO:teuthology.orchestra.run.smithi031.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.0 MB/s | 215 kB 00:00 2024-09-16T00:28:47.045 INFO:teuthology.orchestra.run.smithi081.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 151 kB/s | 16 kB 00:00 2024-09-16T00:28:47.063 INFO:teuthology.orchestra.run.smithi031.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-16T00:28:47.071 INFO:teuthology.orchestra.run.smithi081.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.6 MB/s | 285 kB 00:00 2024-09-16T00:28:47.113 INFO:teuthology.orchestra.run.smithi081.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 440 kB/s | 18 kB 00:00 2024-09-16T00:28:47.138 INFO:teuthology.orchestra.run.smithi031.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-16T00:28:47.188 INFO:teuthology.orchestra.run.smithi081.stdout:(38/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 5.3 MB/s | 765 kB 00:00 2024-09-16T00:28:47.214 INFO:teuthology.orchestra.run.smithi031.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 289 kB/s | 22 kB 00:00 2024-09-16T00:28:47.247 INFO:teuthology.orchestra.run.smithi031.stdout:(51/156): python3-cryptography-36.0.1-4.el9.x86 1.8 MB/s | 1.2 MB 00:00 2024-09-16T00:28:47.281 INFO:teuthology.orchestra.run.smithi031.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-09-16T00:28:47.356 INFO:teuthology.orchestra.run.smithi031.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-09-16T00:28:47.381 INFO:teuthology.orchestra.run.smithi031.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 244 kB/s | 33 kB 00:00 2024-09-16T00:28:47.469 INFO:teuthology.orchestra.run.smithi031.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 127 kB/s | 11 kB 00:00 2024-09-16T00:28:47.490 INFO:teuthology.orchestra.run.smithi031.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 18 kB 00:00 2024-09-16T00:28:47.541 INFO:teuthology.orchestra.run.smithi031.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-16T00:28:47.565 INFO:teuthology.orchestra.run.smithi031.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-16T00:28:47.572 INFO:teuthology.orchestra.run.smithi081.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-09-16T00:28:47.591 INFO:teuthology.orchestra.run.smithi031.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-09-16T00:28:47.666 INFO:teuthology.orchestra.run.smithi031.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 5.7 MB/s | 581 kB 00:00 2024-09-16T00:28:47.691 INFO:teuthology.orchestra.run.smithi031.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 244 kB/s | 24 kB 00:00 2024-09-16T00:28:47.724 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T00:28:47.733 INFO:teuthology.orchestra.run.smithi031.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-16T00:28:47.759 INFO:teuthology.orchestra.run.smithi031.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-09-16T00:28:47.772 INFO:teuthology.orchestra.run.smithi081.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 956 kB/s | 191 kB 00:00 2024-09-16T00:28:47.784 INFO:teuthology.orchestra.run.smithi031.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-16T00:28:47.817 INFO:teuthology.orchestra.run.smithi031.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 658 kB/s | 38 kB 00:00 2024-09-16T00:28:47.842 INFO:teuthology.orchestra.run.smithi031.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-16T00:28:47.874 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T00:28:47.874 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T00:28:47.890 INFO:teuthology.orchestra.run.smithi081.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-16T00:28:47.916 INFO:teuthology.orchestra.run.smithi081.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-16T00:28:47.951 INFO:teuthology.orchestra.run.smithi031.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-09-16T00:28:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-16T00:28:48.025 INFO:teuthology.orchestra.run.smithi081.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 106 kB 00:00 2024-09-16T00:28:48.026 INFO:teuthology.orchestra.run.smithi031.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-16T00:28:48.094 INFO:teuthology.orchestra.run.smithi031.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-16T00:28:48.119 INFO:teuthology.orchestra.run.smithi031.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-16T00:28:48.142 INFO:teuthology.orchestra.run.smithi081.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-16T00:28:48.192 INFO:teuthology.orchestra.run.smithi081.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-09-16T00:28:48.211 INFO:teuthology.orchestra.run.smithi031.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-09-16T00:28:48.236 INFO:teuthology.orchestra.run.smithi031.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 748 kB/s | 87 kB 00:00 2024-09-16T00:28:48.250 INFO:teuthology.orchestra.run.smithi081.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-16T00:28:48.269 INFO:teuthology.orchestra.run.smithi031.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.7 MB/s | 1.0 MB 00:00 2024-09-16T00:28:48.303 INFO:teuthology.orchestra.run.smithi031.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 917 kB/s | 61 kB 00:00 2024-09-16T00:28:48.352 INFO:teuthology.orchestra.run.smithi081.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-09-16T00:28:48.387 INFO:teuthology.orchestra.run.smithi031.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-09-16T00:28:48.412 INFO:teuthology.orchestra.run.smithi031.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-16T00:28:48.434 INFO:teuthology.orchestra.run.smithi081.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-16T00:28:48.437 INFO:teuthology.orchestra.run.smithi031.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-09-16T00:28:48.462 INFO:teuthology.orchestra.run.smithi031.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-09-16T00:28:48.504 INFO:teuthology.orchestra.run.smithi031.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-09-16T00:28:48.518 INFO:teuthology.orchestra.run.smithi081.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 5.6 MB/s | 943 kB 00:00 2024-09-16T00:28:48.530 INFO:teuthology.orchestra.run.smithi031.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 517 kB/s | 35 kB 00:00 2024-09-16T00:28:48.543 INFO:teuthology.orchestra.run.smithi081.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 984 kB/s | 107 kB 00:00 2024-09-16T00:28:48.555 INFO:teuthology.orchestra.run.smithi031.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-16T00:28:48.594 INFO:teuthology.orchestra.run.smithi081.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 289 kB/s | 22 kB 00:00 2024-09-16T00:28:48.597 INFO:teuthology.orchestra.run.smithi031.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-16T00:28:48.622 INFO:teuthology.orchestra.run.smithi031.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-16T00:28:48.656 INFO:teuthology.orchestra.run.smithi031.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-16T00:28:48.661 INFO:teuthology.orchestra.run.smithi081.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-09-16T00:28:48.686 INFO:teuthology.orchestra.run.smithi081.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-09-16T00:28:48.731 INFO:teuthology.orchestra.run.smithi031.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-16T00:28:48.756 INFO:teuthology.orchestra.run.smithi031.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 315 kB/s | 42 kB 00:00 2024-09-16T00:28:48.782 INFO:teuthology.orchestra.run.smithi031.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-09-16T00:28:48.786 INFO:teuthology.orchestra.run.smithi081.stdout:(54/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-16T00:28:48.803 INFO:teuthology.orchestra.run.smithi081.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-16T00:28:48.815 INFO:teuthology.orchestra.run.smithi031.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-16T00:28:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-09-16T00:28:48.854 INFO:teuthology.orchestra.run.smithi081.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-09-16T00:28:48.857 INFO:teuthology.orchestra.run.smithi031.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-16T00:28:48.874 INFO:teuthology.orchestra.run.smithi031.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 913 kB/s | 54 kB 00:00 2024-09-16T00:28:48.896 INFO:teuthology.orchestra.run.smithi081.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-16T00:28:48.916 INFO:teuthology.orchestra.run.smithi031.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-09-16T00:28:48.921 INFO:teuthology.orchestra.run.smithi081.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-09-16T00:28:48.966 INFO:teuthology.orchestra.run.smithi031.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 480 kB/s | 44 kB 00:00 2024-09-16T00:28:48.983 INFO:teuthology.orchestra.run.smithi031.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-16T00:28:49.042 INFO:teuthology.orchestra.run.smithi031.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 479 kB/s | 36 kB 00:00 2024-09-16T00:28:49.059 INFO:teuthology.orchestra.run.smithi031.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-09-16T00:28:49.226 INFO:teuthology.orchestra.run.smithi031.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-09-16T00:28:49.243 INFO:teuthology.orchestra.run.smithi031.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 315 kB/s | 58 kB 00:00 2024-09-16T00:28:49.276 INFO:teuthology.orchestra.run.smithi031.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-16T00:28:49.302 INFO:teuthology.orchestra.run.smithi031.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-16T00:28:49.327 INFO:teuthology.orchestra.run.smithi031.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 759 kB/s | 64 kB 00:00 2024-09-16T00:28:49.352 INFO:teuthology.orchestra.run.smithi031.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-16T00:28:49.402 INFO:teuthology.orchestra.run.smithi031.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-16T00:28:49.413 INFO:teuthology.orchestra.run.smithi081.stdout:(59/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 8.7 MB/s | 49 MB 00:05 2024-09-16T00:28:49.427 INFO:teuthology.orchestra.run.smithi031.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-09-16T00:28:49.455 INFO:teuthology.orchestra.run.smithi081.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 1.1 MB/s | 581 kB 00:00 2024-09-16T00:28:49.461 INFO:teuthology.orchestra.run.smithi031.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 5.2 MB/s | 308 kB 00:00 2024-09-16T00:28:49.480 INFO:teuthology.orchestra.run.smithi081.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.1 MB/s | 662 kB 00:00 2024-09-16T00:28:49.488 INFO:teuthology.orchestra.run.smithi031.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-16T00:28:49.512 INFO:teuthology.orchestra.run.smithi031.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T00:28:49.522 INFO:teuthology.orchestra.run.smithi081.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-09-16T00:28:49.537 INFO:teuthology.orchestra.run.smithi031.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T00:28:49.562 INFO:teuthology.orchestra.run.smithi031.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T00:28:49.564 INFO:teuthology.orchestra.run.smithi081.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-16T00:28:49.589 INFO:teuthology.orchestra.run.smithi081.stdout:(64/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 139 kB/s | 24 kB 00:00 2024-09-16T00:28:49.596 INFO:teuthology.orchestra.run.smithi031.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-16T00:28:49.615 INFO:teuthology.orchestra.run.smithi081.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-16T00:28:49.629 INFO:teuthology.orchestra.run.smithi031.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-16T00:28:49.640 INFO:teuthology.orchestra.run.smithi081.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 510 kB/s | 38 kB 00:00 2024-09-16T00:28:49.676 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T00:28:49.677 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T00:28:49.680 INFO:teuthology.orchestra.run.smithi031.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-16T00:28:49.705 INFO:teuthology.orchestra.run.smithi031.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:28:49.730 INFO:teuthology.orchestra.run.smithi031.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T00:28:49.740 INFO:teuthology.orchestra.run.smithi081.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T00:28:49.755 INFO:teuthology.orchestra.run.smithi031.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 571 kB/s | 14 kB 00:00 2024-09-16T00:28:49.789 INFO:teuthology.orchestra.run.smithi031.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.2 MB/s | 839 kB 00:00 2024-09-16T00:28:49.823 INFO:teuthology.orchestra.run.smithi031.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-16T00:28:49.824 INFO:teuthology.orchestra.run.smithi081.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-16T00:28:49.857 INFO:teuthology.orchestra.run.smithi031.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-16T00:28:49.866 INFO:teuthology.orchestra.run.smithi081.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-16T00:28:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-16T00:28:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-16T00:28:49.941 INFO:teuthology.orchestra.run.smithi081.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 14 MB/s | 4.9 MB 00:00 2024-09-16T00:28:49.967 INFO:teuthology.orchestra.run.smithi081.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 966 kB/s | 137 kB 00:00 2024-09-16T00:28:49.974 INFO:teuthology.orchestra.run.smithi031.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-09-16T00:28:50.010 INFO:teuthology.orchestra.run.smithi031.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 298 kB/s | 11 kB 00:00 2024-09-16T00:28:50.041 INFO:teuthology.orchestra.run.smithi031.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-09-16T00:28:50.059 INFO:teuthology.orchestra.run.smithi081.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-09-16T00:28:50.066 INFO:teuthology.orchestra.run.smithi031.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 316 kB/s | 18 kB 00:00 2024-09-16T00:28:50.084 INFO:teuthology.orchestra.run.smithi081.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 748 kB/s | 87 kB 00:00 2024-09-16T00:28:50.092 INFO:teuthology.orchestra.run.smithi031.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-16T00:28:50.117 INFO:teuthology.orchestra.run.smithi031.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-16T00:28:50.117 INFO:teuthology.orchestra.run.smithi081.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.7 MB/s | 1.0 MB 00:00 2024-09-16T00:28:50.142 INFO:teuthology.orchestra.run.smithi031.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-16T00:28:50.159 INFO:teuthology.orchestra.run.smithi081.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 813 kB/s | 61 kB 00:00 2024-09-16T00:28:50.167 INFO:teuthology.orchestra.run.smithi031.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:28:50.193 INFO:teuthology.orchestra.run.smithi031.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T00:28:50.201 INFO:teuthology.orchestra.run.smithi081.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-16T00:28:50.226 INFO:teuthology.orchestra.run.smithi031.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-16T00:28:50.227 INFO:teuthology.orchestra.run.smithi081.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-16T00:28:50.260 INFO:teuthology.orchestra.run.smithi031.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-16T00:28:50.285 INFO:teuthology.orchestra.run.smithi031.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-16T00:28:50.310 INFO:teuthology.orchestra.run.smithi081.stdout:(78/156): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-09-16T00:28:50.327 INFO:teuthology.orchestra.run.smithi031.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-09-16T00:28:50.335 INFO:teuthology.orchestra.run.smithi081.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-09-16T00:28:50.353 INFO:teuthology.orchestra.run.smithi031.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-16T00:28:50.363 INFO:teuthology.orchestra.run.smithi081.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 140 kB/s | 19 kB 00:00 2024-09-16T00:28:50.369 INFO:teuthology.orchestra.run.smithi031.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 677 kB/s | 28 kB 00:00 2024-09-16T00:28:50.386 INFO:teuthology.orchestra.run.smithi081.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-16T00:28:50.394 INFO:teuthology.orchestra.run.smithi031.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-16T00:28:50.420 INFO:teuthology.orchestra.run.smithi031.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:28:50.436 INFO:teuthology.orchestra.run.smithi081.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 481 kB/s | 35 kB 00:00 2024-09-16T00:28:50.445 INFO:teuthology.orchestra.run.smithi031.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-16T00:28:50.470 INFO:teuthology.orchestra.run.smithi031.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-16T00:28:50.478 INFO:teuthology.orchestra.run.smithi081.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-16T00:28:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-16T00:28:50.520 INFO:teuthology.orchestra.run.smithi081.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-16T00:28:50.537 INFO:teuthology.orchestra.run.smithi031.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-09-16T00:28:50.545 INFO:teuthology.orchestra.run.smithi081.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-16T00:28:50.563 INFO:teuthology.orchestra.run.smithi031.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-16T00:28:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-16T00:28:50.614 INFO:teuthology.orchestra.run.smithi031.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-16T00:28:50.621 INFO:teuthology.orchestra.run.smithi081.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-09-16T00:28:50.637 INFO:teuthology.orchestra.run.smithi081.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-09-16T00:28:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T00:28:50.663 INFO:teuthology.orchestra.run.smithi081.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-16T00:28:50.672 INFO:teuthology.orchestra.run.smithi031.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-16T00:28:50.705 INFO:teuthology.orchestra.run.smithi081.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-16T00:28:50.722 INFO:teuthology.orchestra.run.smithi031.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T00:28:50.730 INFO:teuthology.orchestra.run.smithi081.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.9 MB/s | 279 kB 00:00 2024-09-16T00:28:50.747 INFO:teuthology.orchestra.run.smithi081.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-09-16T00:28:50.781 INFO:teuthology.orchestra.run.smithi081.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-09-16T00:28:50.806 INFO:teuthology.orchestra.run.smithi081.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 710 kB/s | 42 kB 00:00 2024-09-16T00:28:50.865 INFO:teuthology.orchestra.run.smithi081.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 529 kB/s | 44 kB 00:00 2024-09-16T00:28:50.890 INFO:teuthology.orchestra.run.smithi031.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-16T00:28:50.890 INFO:teuthology.orchestra.run.smithi081.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-16T00:28:50.932 INFO:teuthology.orchestra.run.smithi081.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-09-16T00:28:50.957 INFO:teuthology.orchestra.run.smithi081.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 710 kB/s | 47 kB 00:00 2024-09-16T00:28:51.007 INFO:teuthology.orchestra.run.smithi081.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.0 MB/s | 304 kB 00:00 2024-09-16T00:28:51.033 INFO:teuthology.orchestra.run.smithi081.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-09-16T00:28:51.048 INFO:teuthology.orchestra.run.smithi031.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 8.4 MB/s | 3.2 MB 00:00 2024-09-16T00:28:51.091 INFO:teuthology.orchestra.run.smithi081.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-16T00:28:51.125 INFO:teuthology.orchestra.run.smithi081.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 691 kB/s | 64 kB 00:00 2024-09-16T00:28:51.292 INFO:teuthology.orchestra.run.smithi081.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-16T00:28:51.317 INFO:teuthology.orchestra.run.smithi081.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 116 kB/s | 22 kB 00:00 2024-09-16T00:28:51.332 INFO:teuthology.orchestra.run.smithi031.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 144 kB/s | 41 kB 00:00 2024-09-16T00:28:51.343 INFO:teuthology.orchestra.run.smithi081.stdout:(104/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 178 kB/s | 45 kB 00:00 2024-09-16T00:28:51.393 INFO:teuthology.orchestra.run.smithi081.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 8.4 MB/s | 863 kB 00:00 2024-09-16T00:28:51.435 INFO:teuthology.orchestra.run.smithi081.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-09-16T00:28:51.468 INFO:teuthology.orchestra.run.smithi081.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.4 MB/s | 308 kB 00:00 2024-09-16T00:28:51.494 INFO:teuthology.orchestra.run.smithi081.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 197 kB/s | 19 kB 00:00 2024-09-16T00:28:51.519 INFO:teuthology.orchestra.run.smithi081.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-09-16T00:28:51.544 INFO:teuthology.orchestra.run.smithi081.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-16T00:28:51.570 INFO:teuthology.orchestra.run.smithi081.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T00:28:51.595 INFO:teuthology.orchestra.run.smithi081.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-16T00:28:51.637 INFO:teuthology.orchestra.run.smithi081.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-16T00:28:51.640 INFO:teuthology.orchestra.run.smithi031.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 2.4 MB/s | 1.8 MB 00:00 2024-09-16T00:28:51.662 INFO:teuthology.orchestra.run.smithi081.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:28:51.687 INFO:teuthology.orchestra.run.smithi081.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-16T00:28:51.704 INFO:teuthology.orchestra.run.smithi081.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 768 kB/s | 32 kB 00:00 2024-09-16T00:28:51.721 INFO:teuthology.orchestra.run.smithi081.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 423 kB/s | 14 kB 00:00 2024-09-16T00:28:51.747 INFO:teuthology.orchestra.run.smithi081.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-16T00:28:51.781 INFO:teuthology.orchestra.run.smithi081.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-16T00:28:51.799 INFO:teuthology.orchestra.run.smithi031.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 533 kB/s | 249 kB 00:00 2024-09-16T00:28:51.805 INFO:teuthology.orchestra.run.smithi081.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-16T00:28:51.831 INFO:teuthology.orchestra.run.smithi081.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-16T00:28:51.865 INFO:teuthology.orchestra.run.smithi081.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-16T00:28:51.865 INFO:teuthology.orchestra.run.smithi031.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 1.4 MB 00:00 2024-09-16T00:28:51.890 INFO:teuthology.orchestra.run.smithi081.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-16T00:28:51.915 INFO:teuthology.orchestra.run.smithi081.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-16T00:28:51.940 INFO:teuthology.orchestra.run.smithi081.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-16T00:28:51.966 INFO:teuthology.orchestra.run.smithi081.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-16T00:28:51.991 INFO:teuthology.orchestra.run.smithi081.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-16T00:28:52.066 INFO:teuthology.orchestra.run.smithi081.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-09-16T00:28:52.117 INFO:teuthology.orchestra.run.smithi081.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-09-16T00:28:52.132 INFO:teuthology.orchestra.run.smithi031.stdout:(155/156): NetworkManager-1.49.90-1.el9.x86_64. 1.7 MB/s | 2.3 MB 00:01 2024-09-16T00:28:52.142 INFO:teuthology.orchestra.run.smithi081.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 613 kB/s | 46 kB 00:00 2024-09-16T00:28:52.167 INFO:teuthology.orchestra.run.smithi081.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-16T00:28:52.192 INFO:teuthology.orchestra.run.smithi081.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-16T00:28:52.218 INFO:teuthology.orchestra.run.smithi081.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-16T00:28:52.243 INFO:teuthology.orchestra.run.smithi081.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-16T00:28:52.277 INFO:teuthology.orchestra.run.smithi081.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-16T00:28:52.291 INFO:teuthology.orchestra.run.smithi031.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.9 MB/s | 2.4 MB 00:00 2024-09-16T00:28:52.298 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:28:52.298 INFO:teuthology.orchestra.run.smithi031.stdout:Total 21 MB/s | 226 MB 00:11 2024-09-16T00:28:52.302 INFO:teuthology.orchestra.run.smithi081.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-09-16T00:28:52.327 INFO:teuthology.orchestra.run.smithi081.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-16T00:28:52.353 INFO:teuthology.orchestra.run.smithi081.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:28:52.378 INFO:teuthology.orchestra.run.smithi081.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-16T00:28:52.403 INFO:teuthology.orchestra.run.smithi081.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-16T00:28:52.437 INFO:teuthology.orchestra.run.smithi081.stdout:(141/156): python3-kubernetes-26.1.0-2.el9.noar 2.3 MB/s | 1.0 MB 00:00 2024-09-16T00:28:52.470 INFO:teuthology.orchestra.run.smithi081.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-16T00:28:52.496 INFO:teuthology.orchestra.run.smithi081.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-16T00:28:52.521 INFO:teuthology.orchestra.run.smithi081.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-16T00:28:52.546 INFO:teuthology.orchestra.run.smithi081.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-16T00:28:52.572 INFO:teuthology.orchestra.run.smithi081.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T00:28:52.597 INFO:teuthology.orchestra.run.smithi081.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-16T00:28:52.639 INFO:teuthology.orchestra.run.smithi081.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-16T00:28:52.973 INFO:teuthology.orchestra.run.smithi081.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 8.4 MB/s | 3.2 MB 00:00 2024-09-16T00:28:53.265 INFO:teuthology.orchestra.run.smithi081.stdout:(150/156): librados2-19.1.1-326.g8921c8ef.el9.x 4.9 MB/s | 3.4 MB 00:00 2024-09-16T00:28:53.398 INFO:teuthology.orchestra.run.smithi081.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 3.1 MB/s | 2.3 MB 00:00 2024-09-16T00:28:53.515 INFO:teuthology.orchestra.run.smithi081.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-16T00:28:53.540 INFO:teuthology.orchestra.run.smithi081.stdout:(153/156): NetworkManager-team-1.49.90-1.el9.x8 148 kB/s | 41 kB 00:00 2024-09-16T00:28:53.573 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T00:28:53.657 INFO:teuthology.orchestra.run.smithi081.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-09-16T00:28:53.732 INFO:teuthology.orchestra.run.smithi081.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 2.4 MB/s | 1.8 MB 00:00 2024-09-16T00:28:53.741 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T00:28:53.741 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T00:28:54.199 INFO:teuthology.orchestra.run.smithi081.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 2.4 MB 00:00 2024-09-16T00:28:54.206 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:28:54.206 INFO:teuthology.orchestra.run.smithi081.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-16T00:28:55.449 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-16T00:28:55.597 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-16T00:28:55.597 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-16T00:28:55.925 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T00:28:56.053 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:28:56.064 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T00:28:56.065 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T00:28:56.091 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:28:56.144 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:28:56.178 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:28:56.204 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:28:56.254 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:28:56.638 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:28:56.722 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:28:56.843 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:28:56.873 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:28:56.963 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:28:56.993 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:28:57.138 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:28:57.199 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:28:57.226 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:28:57.314 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:28:57.336 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:28:57.398 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:28:57.406 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-16T00:28:57.407 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-16T00:28:57.421 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:28:57.443 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:28:57.630 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:28:57.780 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:28:57.893 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:28:57.985 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:28:58.055 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:28:58.270 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:28:58.379 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:28:58.473 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:28:58.513 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:28:58.548 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:28:58.621 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:28:58.673 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:28:58.705 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:28:58.749 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:28:58.781 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:28:58.809 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:28:58.877 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:28:58.936 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:28:58.971 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:28:59.024 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:28:59.100 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:28:59.143 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:28:59.186 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:28:59.215 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:28:59.285 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:28:59.330 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:28:59.404 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:28:59.542 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:28:59.784 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:29:00.409 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:29:00.493 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:29:00.531 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:29:00.575 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:29:00.622 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:29:00.680 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:29:00.837 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:29:00.916 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:29:00.971 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:29:01.044 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:29:01.076 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:29:01.112 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:29:01.167 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:29:01.211 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:29:01.254 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:29:01.297 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:29:01.431 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:29:01.473 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:29:01.515 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:29:01.615 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:29:01.835 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:29:01.915 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:29:01.948 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:29:01.973 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:29:02.051 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:29:02.173 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:29:02.205 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:29:02.232 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:29:02.292 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:29:02.340 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:29:02.371 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:29:02.401 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:29:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T00:29:02.435 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:29:02.469 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:29:02.510 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:29:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:29:02.545 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:29:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:29:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:29:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:29:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:29:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:29:02.866 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:29:02.872 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:29:02.913 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:29:02.946 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:29:03.010 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:29:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:29:03.470 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:29:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:29:03.588 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:29:03.613 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:29:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:29:03.640 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-16T00:29:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:29:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:29:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:29:03.753 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:29:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:29:03.792 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:29:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:29:03.845 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:29:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:29:03.911 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:29:03.912 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:29:03.938 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:29:03.942 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:29:03.973 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:29:03.977 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:29:03.980 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:29:04.000 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:29:04.106 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:29:04.135 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:29:04.161 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:29:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:29:04.200 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:29:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:29:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:29:04.276 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:29:04.311 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:29:04.448 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:29:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:29:04.608 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:29:04.734 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:29:04.838 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:29:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:29:04.873 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:29:04.917 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:29:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:29:04.965 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:29:04.995 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:29:05.032 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:29:05.146 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:29:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:29:05.199 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:29:05.227 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:29:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:29:05.353 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:29:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:29:05.416 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:29:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:29:05.438 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:29:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:29:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:29:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:29:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:29:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:29:05.639 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:29:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:29:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:29:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:29:05.777 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:29:05.801 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:29:05.852 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:29:05.898 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:29:05.919 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:29:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:29:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:29:06.029 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:29:06.052 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:29:06.096 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:29:06.104 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:29:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:29:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:29:06.237 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:29:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:29:06.326 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:29:06.329 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:29:06.334 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:29:06.356 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:29:06.390 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:29:06.447 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:29:06.470 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:29:06.526 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:29:06.555 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:29:06.584 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:29:06.619 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:29:06.692 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:29:06.699 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:29:06.744 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:29:06.745 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:29:06.780 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:29:06.832 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:29:06.834 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:29:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:29:06.868 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:29:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:29:06.902 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:29:06.962 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:29:07.016 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:29:07.046 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:29:07.081 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:29:07.093 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:29:07.160 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:29:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:29:07.192 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:29:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:29:07.244 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:29:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:29:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:29:07.281 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:29:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:29:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:29:07.383 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:29:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:29:07.439 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:29:07.469 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:29:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:29:07.510 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:29:07.541 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:29:07.557 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:29:07.602 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:29:07.614 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:29:07.656 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:29:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:29:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:29:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:29:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:29:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:29:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:29:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:29:08.181 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:29:08.256 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:29:08.261 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:29:08.290 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:29:08.297 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:29:08.333 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:29:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:29:08.371 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:29:08.413 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:29:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:29:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:29:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:29:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:29:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:29:08.586 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:29:08.651 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:29:08.670 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:29:08.714 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:29:08.786 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:29:08.803 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:29:08.819 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:29:08.836 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:29:08.855 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:29:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:29:08.901 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:29:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:29:08.937 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:29:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:29:08.980 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:29:09.002 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:29:09.032 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:29:09.047 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:29:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:29:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:29:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:29:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:29:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:29:09.233 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:29:09.251 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:29:09.285 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:29:09.301 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:29:09.332 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:29:09.388 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:29:09.418 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:29:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:29:09.458 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:29:09.484 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:29:09.509 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:29:09.584 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:29:09.673 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:29:09.709 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:29:09.741 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:29:09.769 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:29:09.811 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:29:09.841 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:29:09.871 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:29:10.053 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:29:10.092 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:29:10.122 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:29:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:29:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:29:10.232 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:29:10.265 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:29:10.357 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:29:10.405 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:29:10.420 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:29:10.444 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:29:10.458 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:29:10.549 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:29:10.549 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:29:10.549 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:29:10.549 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:10.550 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:29:10.576 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:29:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:29:10.602 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:29:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:29:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:29:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:29:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:29:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:29:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:29:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:29:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:29:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:29:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:29:11.315 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:29:11.375 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:29:11.408 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:29:11.507 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:29:11.564 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:29:11.608 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:29:11.696 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:29:11.737 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:29:11.921 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:29:11.951 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:29:11.997 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:29:12.027 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:29:12.123 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:29:12.668 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:29:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:29:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:29:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:29:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:29:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:29:14.000 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:29:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:29:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:29:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:29:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:29:14.141 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:29:14.450 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:29:14.504 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:29:14.594 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:29:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:29:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:29:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:29:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:29:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:29:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:29:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:29:15.244 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:29:15.267 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:29:17.333 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:29:17.374 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:29:17.886 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:29:17.922 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:29:17.954 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:29:18.094 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:29:18.094 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:29:18.095 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:29:18.095 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:18.137 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:29:18.473 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:29:18.519 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:29:18.603 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:29:18.604 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:29:18.604 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:29:18.604 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:18.645 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:29:22.345 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:29:22.345 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-09-16T00:29:22.345 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-09-16T00:29:22.345 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-09-16T00:29:22.345 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-09-16T00:29:22.345 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-09-16T00:29:22.345 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-09-16T00:29:22.345 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-09-16T00:29:22.345 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:22.521 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:29:22.539 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:29:22.539 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:22.540 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:29:22.540 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:23.999 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:29:24.026 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:29:24.026 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:24.026 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:29:24.026 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:29:24.026 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:29:24.026 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:24.250 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:29:24.273 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:29:24.273 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:24.273 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:29:24.273 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:29:24.274 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:29:24.274 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:24.709 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:29:24.732 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:29:24.732 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:24.732 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:29:24.732 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:29:24.732 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:29:24.732 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:24.839 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:29:24.888 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:29:24.951 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:29:25.134 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:29:25.193 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:29:25.271 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:29:25.359 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:29:25.488 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:29:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:29:25.575 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:29:25.611 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:29:25.999 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:29:26.026 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:29:26.059 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:29:26.120 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:29:26.150 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:29:27.182 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:29:27.226 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:29:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:29:28.218 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:29:28.319 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:29:28.481 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:29:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:29:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:29:28.528 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:28.528 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:29:28.528 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:29:28.528 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:29:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:28.580 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:29:28.591 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:29:28.688 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:29:28.753 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:29:29.673 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:29:29.707 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:29:29.731 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:29:29.731 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:29.731 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:29:29.731 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:29:29.731 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:29:29.732 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:29.775 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:29:29.797 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:29:29.797 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:29.797 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:29:29.797 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:29.842 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:29:29.865 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:29:29.865 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:29.865 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:29:29.865 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:29:29.866 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:29:29.866 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:29:30.192 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:29:30.192 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:30.192 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:29:30.192 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:29:30.192 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:29:30.192 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:30.311 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:29:30.312 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2024-09-16T00:29:30.312 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2024-09-16T00:29:30.312 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2024-09-16T00:29:30.312 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2024-09-16T00:29:30.312 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2024-09-16T00:29:30.312 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2024-09-16T00:29:30.312 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2024-09-16T00:29:30.312 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:30.483 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:29:30.499 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:29:30.499 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:30.499 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:29:30.500 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:29:30.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-09-16T00:29:30.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-09-16T00:29:30.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-09-16T00:29:30.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-09-16T00:29:30.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-09-16T00:29:30.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-09-16T00:29:30.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-09-16T00:29:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:30.972 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:29:30.989 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:29:30.989 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:30.989 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:29:30.989 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:31.986 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:29:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:29:32.013 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:32.013 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:29:32.013 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:29:32.013 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:29:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:29:32.260 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:29:32.260 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:32.261 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:29:32.261 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:29:32.261 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:29:32.261 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:29:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:29:32.545 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:32.545 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:29:32.545 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:29:32.545 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:29:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:29:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:29:32.719 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:32.719 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:29:32.719 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:29:32.720 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:29:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:29:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:29:32.809 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:32.809 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:29:32.809 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:29:32.809 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:29:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:32.825 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:29:32.882 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:29:32.937 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:29:33.122 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:29:33.171 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:29:33.257 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:29:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:29:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:29:33.293 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:33.293 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:29:33.293 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:29:33.293 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:29:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:33.345 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:29:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:29:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:29:33.484 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:29:33.521 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:29:33.530 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:29:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:29:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:29:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:29:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:29:33.841 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:29:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:29:34.077 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:29:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:29:34.113 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:29:34.144 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:29:34.152 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:29:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:29:34.203 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:29:34.215 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:29:34.244 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:29:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:29:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:29:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:29:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:29:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:29:34.721 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:29:34.762 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:29:34.803 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:29:34.836 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:29:34.863 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:29:35.043 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:29:35.086 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:29:35.115 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:29:35.144 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:29:35.199 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:29:35.278 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:29:35.313 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:29:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:29:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:29:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:29:35.855 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:29:35.876 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:29:36.271 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:29:36.297 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:29:36.398 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:29:36.560 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:29:36.590 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:29:36.615 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:29:36.615 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:36.615 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:29:36.616 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:29:36.616 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:29:36.616 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:36.666 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:29:36.678 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:29:36.704 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:29:36.775 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:29:36.785 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:29:36.830 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:29:36.835 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:29:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:29:36.884 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:29:36.924 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:29:36.963 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:29:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:29:37.006 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:29:37.031 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:29:37.032 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:29:37.116 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:29:37.116 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:29:37.166 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:29:37.195 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:29:37.200 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:29:37.220 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:29:37.220 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:37.221 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:29:37.221 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:29:37.221 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:29:37.221 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:37.233 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:29:37.238 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:29:37.238 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:29:37.270 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:29:37.283 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:29:37.376 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:29:37.418 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:29:37.419 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:29:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:29:37.503 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:29:37.565 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:29:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:29:37.809 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:29:37.809 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:37.810 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:29:37.810 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:29:37.810 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:29:37.810 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:29:37.876 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:29:37.876 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:37.876 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:29:37.877 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:37.928 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:29:37.952 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:29:37.953 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:37.953 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:29:37.953 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:29:37.953 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:29:37.953 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:38.246 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:29:38.269 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:29:38.269 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:38.269 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:29:38.269 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:29:38.269 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:29:38.269 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:29:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:29:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:29:38.465 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:38.465 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:29:38.465 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:29:38.465 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:29:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:29:38.531 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:29:38.532 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:38.532 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:29:38.532 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:38.582 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:29:38.607 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:29:38.607 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:38.607 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:29:38.607 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:29:38.608 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:29:38.608 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:29:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:29:38.950 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:29:38.950 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:29:38.950 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:29:38.950 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:29:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:42.816 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:29:42.857 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:29:42.898 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:29:42.930 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:29:42.957 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:29:43.139 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:29:43.139 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:29:43.140 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:29:43.141 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:29:43.141 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:29:43.141 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:29:43.141 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:29:43.142 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:29:43.143 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:29:43.143 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:29:43.151 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:29:43.156 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:29:43.156 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:29:43.156 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:29:43.156 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:29:43.156 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:29:43.156 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:29:43.157 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:29:43.157 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:29:43.157 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:29:43.157 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:29:43.157 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:29:43.157 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:29:43.157 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:29:43.157 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:29:43.157 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:29:43.158 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:29:43.159 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:29:43.159 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:29:43.159 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:29:43.159 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:29:43.159 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:29:43.159 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:29:43.159 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:29:43.159 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:29:43.159 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:29:43.160 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:29:43.160 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:29:43.160 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:29:43.160 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:29:43.160 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:29:43.160 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:29:43.160 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:29:43.160 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:29:43.160 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:29:43.161 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:29:43.161 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:29:43.161 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:29:43.161 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:29:43.161 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:29:43.161 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:29:43.162 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:29:43.162 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:29:43.162 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:29:43.162 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:29:43.162 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:29:43.162 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:29:43.163 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:29:43.163 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:29:43.163 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:29:43.163 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:29:43.163 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:29:43.163 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:29:43.163 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:29:43.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:29:43.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:29:43.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:29:43.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:29:43.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:29:43.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:29:43.165 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:29:43.165 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:29:43.165 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:29:43.165 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:29:43.165 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:29:43.165 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:29:43.165 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:29:43.166 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:29:43.166 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:29:43.166 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:29:43.166 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:29:43.166 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:29:43.166 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:29:43.167 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:29:43.167 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:29:43.167 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:29:43.167 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:29:43.167 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:29:43.167 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:29:43.167 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:29:43.167 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:29:43.167 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:29:43.168 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:29:43.168 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:29:43.168 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:29:43.168 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:29:43.168 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:29:43.168 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:29:43.168 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:29:43.169 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:29:43.169 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:29:43.169 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:29:43.169 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:29:43.169 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:29:43.169 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:29:43.169 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:29:43.169 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:29:43.169 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:29:43.170 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:29:43.170 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:29:43.170 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:29:43.170 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:29:43.170 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:29:43.170 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:29:43.170 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:29:43.170 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:29:43.171 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:29:43.171 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:29:43.171 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:29:43.171 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:29:43.171 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:29:43.171 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:29:43.171 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:29:43.171 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:29:43.172 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:29:43.172 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:29:43.172 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:29:43.172 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:29:43.172 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:29:43.172 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:29:43.172 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:29:43.172 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:29:43.172 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:29:43.173 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:29:43.188 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:29:43.218 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:29:43.247 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:29:43.302 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:29:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:29:43.828 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:29:43.869 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:29:43.873 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:29:43.900 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:29:43.908 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:29:43.928 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:29:43.929 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:29:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:29:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:29:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:29:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:29:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:29:44.767 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:29:44.855 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:29:44.892 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:29:44.896 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:29:44.944 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:29:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:29:44.985 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:29:45.025 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:29:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:29:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:29:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:29:45.185 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:29:45.185 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:29:45.277 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:29:45.302 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:29:45.308 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:29:45.308 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:29:45.488 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:29:45.488 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:29:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:29:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:29:45.892 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:29:45.934 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:29:45.934 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:29:45.935 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.936 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.937 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.938 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:29:45.939 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:29:45.940 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:29:45.941 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:29:45.942 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:29:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:29:45.944 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:29:45.944 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:29:45.944 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:29:45.944 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:29:45.944 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:29:45.944 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:29:45.945 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:29:45.945 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:29:45.945 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:29:45.945 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:29:45.945 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:29:45.945 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:29:45.945 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:29:45.946 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:45.947 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:29:45.948 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:29:45.948 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:29:45.948 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:29:45.948 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:29:45.948 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:29:45.949 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:29:45.949 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:29:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T00:29:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:29:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:29:46.109 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:29:46.143 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:29:46.176 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:29:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:29:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:29:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:29:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:29:46.362 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:29:46.387 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:29:46.393 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:29:46.393 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:29:46.399 DEBUG:teuthology.parallel:result is None 2024-09-16T00:29:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:29:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:29:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:29:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:29:51.373 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:29:51.374 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:29:51.375 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:29:51.376 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:29:51.376 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:29:51.376 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:29:51.376 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:29:51.389 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:29:51.389 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:29:51.389 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:29:51.389 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:29:51.389 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:29:51.389 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:29:51.390 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:29:51.391 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:29:51.392 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:29:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:29:51.394 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:29:51.395 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:29:51.396 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:29:51.397 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:29:51.398 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:29:51.399 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:29:51.400 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:29:51.400 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:29:51.400 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:29:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:29:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:29:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:29:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:29:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:29:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:29:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:29:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:29:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:29:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:29:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:29:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:29:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:29:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:29:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:29:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:29:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:29:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:29:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:29:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:29:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:29:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:29:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:29:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:29:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:29:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:29:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:29:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:29:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:29:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:29:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:29:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:29:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:29:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:29:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:29:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:29:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:29:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:29:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:29:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:29:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:29:53.900 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:29:53.900 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:53.900 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-09-16T00:29:53.900 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:53.900 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:53.900 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:53.900 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.901 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.902 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:29:53.903 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:29:53.904 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:29:53.905 INFO:teuthology.orchestra.run.smithi081.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:29:53.906 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:29:53.907 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:29:53.908 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:29:53.909 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:29:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:29:53.911 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.912 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.913 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:53.913 INFO:teuthology.orchestra.run.smithi081.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:29:53.913 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:29:53.913 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:29:53.913 INFO:teuthology.orchestra.run.smithi081.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:29:53.913 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:29:53.913 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:29:53.914 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:29:53.914 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:29:53.914 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-16T00:29:54.336 DEBUG:teuthology.parallel:result is None 2024-09-16T00:29:54.755 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:29:54.756 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:54.756 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-09-16T00:29:54.756 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:54.756 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:54.756 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:54.756 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:29:54.756 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.756 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.756 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.757 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.758 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:29:54.759 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:29:54.760 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:29:54.761 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:29:54.762 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:29:54.763 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:29:54.764 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:29:54.765 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.766 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:29:54.767 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:29:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:29:54.769 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:29:54.769 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:29:54.769 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:29:54.769 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T00:29:55.132 DEBUG:teuthology.parallel:result is None 2024-09-16T00:29:55.134 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:55.284 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:29:55.308 INFO:teuthology.orchestra.run.smithi031.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:29:55.309 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:29:55.309 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:29:55.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:55.454 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:29:55.481 INFO:teuthology.orchestra.run.smithi081.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:29:55.481 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:29:55.481 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:29:55.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:55.622 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:29:55.647 INFO:teuthology.orchestra.run.smithi102.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:29:55.648 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:29:55.648 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:29:55.650 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T00:29:55.650 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:29:55.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:29:55.682 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:29:55.682 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:29:55.715 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:29:55.715 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:29:55.746 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T00:29:55.746 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:29:55.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:29:55.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:29:55.841 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:29:55.841 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:29:55.871 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:29:55.939 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:29:55.939 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:29:55.968 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:29:56.036 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T00:29:56.037 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:29:56.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:29:56.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:29:56.134 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:29:56.134 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:29:56.162 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:29:56.229 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:29:56.230 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:29:56.259 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:29:56.328 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T00:29:56.328 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:29:56.328 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:29:56.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:29:56.425 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:29:56.425 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:29:56.453 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:29:56.520 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:29:56.520 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:29:56.549 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:29:56.618 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T00:29:56.709 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2}} 2024-09-16T00:29:56.710 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:56.710 INFO:tasks.cephadm:Cluster fsid is ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:29:56.710 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T00:29:56.710 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.31', 'mon.b': '172.21.15.81', 'mon.c': '172.21.15.102'} 2024-09-16T00:29:56.710 INFO:tasks.cephadm:First mon is mon.a on smithi031 2024-09-16T00:29:56.711 INFO:tasks.cephadm:First mgr is x 2024-09-16T00:29:56.711 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T00:29:56.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-09-16T00:29:56.742 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname $(hostname -s) 2024-09-16T00:29:56.770 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2024-09-16T00:29:56.800 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T00:29:56.800 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:56.941 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T00:29:57.083 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:57.085 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T00:29:57.085 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T00:29:57.085 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:29:57.476 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:29 /home/ubuntu/cephtest/cephadm 2024-09-16T00:29:57.476 DEBUG:teuthology.orchestra.run.smithi081:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:29:57.868 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:29 /home/ubuntu/cephtest/cephadm 2024-09-16T00:29:57.869 DEBUG:teuthology.orchestra.run.smithi102:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:29:58.257 INFO:teuthology.orchestra.run.smithi102.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:29 /home/ubuntu/cephtest/cephadm 2024-09-16T00:29:58.257 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:29:58.278 DEBUG:teuthology.orchestra.run.smithi081:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:29:58.298 DEBUG:teuthology.orchestra.run.smithi102:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:29:58.336 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-16T00:29:58.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:29:58.339 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:29:58.341 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:29:59.901 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:29:59.938 INFO:teuthology.orchestra.run.smithi081.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:29:59.973 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:30:30.564 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-16T00:30:30.565 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:30:30.565 INFO:teuthology.orchestra.run.smithi081.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:30:30.565 INFO:teuthology.orchestra.run.smithi081.stdout: "repo_digests": [ 2024-09-16T00:30:30.565 INFO:teuthology.orchestra.run.smithi081.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:30:30.565 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-16T00:30:30.565 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-16T00:30:52.324 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T00:30:52.324 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:30:52.324 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:30:52.324 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2024-09-16T00:30:52.324 INFO:teuthology.orchestra.run.smithi102.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:30:52.324 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-16T00:30:52.324 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T00:30:55.071 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:30:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:30:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:30:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-09-16T00:30:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:30:55.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:30:55.072 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:30:55.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-09-16T00:30:55.156 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph 2024-09-16T00:30:55.190 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2024-09-16T00:30:55.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-09-16T00:30:55.248 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /etc/ceph 2024-09-16T00:30:55.278 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2024-09-16T00:30:55.306 INFO:tasks.cephadm:Writing seed config... 2024-09-16T00:30:55.307 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-16T00:30:55.307 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-16T00:30:55.307 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T00:30:55.307 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-16T00:30:55.307 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-16T00:30:55.307 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-16T00:30:55.308 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T00:30:55.309 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T00:30:55.310 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-16T00:30:55.311 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:30:55.311 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T00:30:55.327 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ccb601ea-73c2-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-16T00:30:55.328 DEBUG:teuthology.orchestra.run.smithi031:mon.a> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T00:30:55.370 DEBUG:teuthology.orchestra.run.smithi031:mgr.x> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T00:30:55.413 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T00:30:55.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:30:55.621 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:30:55.621 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', 'ccb601ea-73c2-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-09-16T00:30:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T00:30:55.621 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-09-16T00:30:55.646 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:30:55.646 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-09-16T00:30:55.646 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-09-16T00:30:55.654 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:30:55.654 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:30:55.661 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:30:55.661 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T00:30:55.667 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-16T00:30:55.674 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-16T00:30:55.674 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:30:55.674 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-09-16T00:30:55.697 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:30:55.697 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T00:30:55.697 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-09-16T00:30:55.697 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-09-16T00:30:55.704 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:30:55.704 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:30:55.710 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:30:55.710 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T00:30:55.717 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-16T00:30:55.723 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-16T00:30:55.723 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:30:55.724 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-09-16T00:30:55.724 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:30:55.724 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 140179435787264 on /run/cephadm/ccb601ea-73c2-11ef-bceb-c7b262605968.lock 2024-09-16T00:30:55.724 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 140179435787264 acquired on /run/cephadm/ccb601ea-73c2-11ef-bceb-c7b262605968.lock 2024-09-16T00:30:55.724 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-09-16T00:30:55.725 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-09-16T00:30:55.725 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-16T00:30:55.728 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-09-16T00:30:55.728 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-09-16T00:30:55.731 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T00:30:55.731 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-16T00:30:55.731 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-09-16T00:30:55.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T00:30:55.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T00:30:55.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:30:55.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-16T00:30:55.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-16T00:30:55.734 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:30:55.735 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-16T00:30:55.735 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-16T00:30:55.735 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T00:30:55.735 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T00:30:55.736 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:30:56.182 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T00:30:56.182 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:30:56.183 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T00:30:56.183 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-16T00:30:56.183 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T00:30:56.183 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T00:30:56.183 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T00:30:57.070 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T00:30:57.071 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T00:30:57.071 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T00:30:58.060 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-09-16T00:30:58.060 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-09-16T00:30:59.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQBCfOdmCWOGFhAAp8v11aW4nspE+enSY/euLQ== 2024-09-16T00:31:00.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQBDfOdmHMJgFBAAzSZEjuFOKIevur+4G5gPWQ== 2024-09-16T00:31:00.969 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQBEfOdm/WzDFBAA0j3KB+pfdehbdK1sbVMXDw== 2024-09-16T00:31:00.970 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-09-16T00:31:01.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:31:01.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-16T00:31:01.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:01.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:31:01.902 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for a [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:31:01.902 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-09-16T00:31:01.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:01.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:31:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:31:01.902 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-09-16T00:31:02.983 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.a on 2024-09-16T00:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T00:31:03.388 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T00:31:03.580 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968.target. 2024-09-16T00:31:03.580 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968.target. 2024-09-16T00:31:03.882 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.a 2024-09-16T00:31:03.882 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T00:31:04.063 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968.target.wants/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@.service. 2024-09-16T00:31:04.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:04 smithi031 systemd[1]: Starting Ceph mon.a for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:31:04.480 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:04 smithi031 podman[30573]: 2024-09-16 00:31:04.39902004 +0000 UTC m=+0.112945082 container create 0c56ea5db5bcb8a0d5236e7c933bac11cb336e71c4779be81ab8cd7708baf0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-16T00:31:04.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:04 smithi031 podman[30573]: 2024-09-16 00:31:04.302617117 +0000 UTC m=+0.016542161 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:31:04.660 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:31:04.660 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-16T00:31:04.668 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:31:04.668 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T00:31:04.668 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-16T00:31:04.668 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:31:04.668 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-09-16T00:31:04.668 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-09-16T00:31:04.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:04 smithi031 podman[30573]: 2024-09-16 00:31:04.576304787 +0000 UTC m=+0.290229828 container init 0c56ea5db5bcb8a0d5236e7c933bac11cb336e71c4779be81ab8cd7708baf0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:31:04.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:04 smithi031 podman[30573]: 2024-09-16 00:31:04.579683857 +0000 UTC m=+0.293608902 container start 0c56ea5db5bcb8a0d5236e7c933bac11cb336e71c4779be81ab8cd7708baf0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:31:04.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:04 smithi031 ceph-mon[30592]: mkfs ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:04.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:04 smithi031 ceph-mon[30592]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:31:04.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:04 smithi031 bash[30573]: 0c56ea5db5bcb8a0d5236e7c933bac11cb336e71c4779be81ab8cd7708baf0e5 2024-09-16T00:31:04.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:04 smithi031 systemd[1]: Started Ceph mon.a for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:31:05.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:31:05.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: monmap epoch 1 2024-09-16T00:31:05.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:05.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: last_changed 2024-09-16T00:31:01.292640+0000 2024-09-16T00:31:05.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:31:05.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: min_mon_release 19 (squid) 2024-09-16T00:31:05.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: election_strategy: 1 2024-09-16T00:31:05.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:31:05.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: fsmap 2024-09-16T00:31:05.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T00:31:05.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: mgrmap e1: no daemons active 2024-09-16T00:31:05.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:05 smithi031 ceph-mon[30592]: from='client.? 172.21.15.31:0/1324461583' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T00:31:05.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T00:31:05.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:05.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T00:31:05.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:05.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-09-16T00:31:05.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.520352s) 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.067+0000 7f66be76a640 1 Processor -- start 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.067+0000 7f66be76a640 1 -- start start 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.067+0000 7f66be76a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b810a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.067+0000 7f66be76a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66b807be00 con 0x7f66b8107fb0 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.067+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b810a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.067+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b810a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37708/0 (socket says 172.21.15.31:37708) 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.068+0000 7f66b7fff640 1 -- 172.21.15.31:0/2033595394 learned_addr learned my addr 172.21.15.31:0/2033595394 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.068+0000 7f66b7fff640 1 -- 172.21.15.31:0/2033595394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b810a8e0 con 0x7f66b8107fb0 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.070+0000 7f66b7fff640 1 --2- 172.21.15.31:0/2033595394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b810a3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009820 tx=0x7f66a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea7d65edf3954ce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.070+0000 7f66b6ffd640 1 -- 172.21.15.31:0/2033595394 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66a8037070 con 0x7f66b8107fb0 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.070+0000 7f66b6ffd640 1 -- 172.21.15.31:0/2033595394 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a8032440 con 0x7f66b8107fb0 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.070+0000 7f66b6ffd640 1 -- 172.21.15.31:0/2033595394 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66a8032740 con 0x7f66b8107fb0 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.070+0000 7f66be76a640 1 -- 172.21.15.31:0/2033595394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 msgr2=0x7f66b810a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.070+0000 7f66be76a640 1 --2- 172.21.15.31:0/2033595394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b810a3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009820 tx=0x7f66a802a270 comp rx=0 tx=0).stop 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.071+0000 7f66be76a640 1 -- 172.21.15.31:0/2033595394 shutdown_connections 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.071+0000 7f66be76a640 1 --2- 172.21.15.31:0/2033595394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b810a3a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.071+0000 7f66be76a640 1 -- 172.21.15.31:0/2033595394 >> 172.21.15.31:0/2033595394 conn(0x7f66b8101b90 msgr2=0x7f66b8103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.071+0000 7f66be76a640 1 -- 172.21.15.31:0/2033595394 shutdown_connections 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.071+0000 7f66be76a640 1 -- 172.21.15.31:0/2033595394 wait complete. 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.071+0000 7f66be76a640 1 Processor -- start 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.071+0000 7f66be76a640 1 -- start start 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66be76a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66be76a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66b810ad50 con 0x7f66b8107fb0 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37716/0 (socket says 172.21.15.31:37716) 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66b7fff640 1 -- 172.21.15.31:0/1324461583 learned_addr learned my addr 172.21.15.31:0/1324461583 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66b7fff640 1 -- 172.21.15.31:0/1324461583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b819b1d0 con 0x7f66b8107fb0 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66b7fff640 1 --2- 172.21.15.31:0/1324461583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b819ac90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f66a80090b0 tx=0x7f66a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66b57fa640 1 -- 172.21.15.31:0/1324461583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66a8041070 con 0x7f66b8107fb0 2024-09-16T00:31:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66b57fa640 1 -- 172.21.15.31:0/1324461583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a803c7a0 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66be76a640 1 -- 172.21.15.31:0/1324461583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b819b460 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66b57fa640 1 -- 172.21.15.31:0/1324461583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66a8037040 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.072+0000 7f66be76a640 1 -- 172.21.15.31:0/1324461583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b819e150 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.073+0000 7f66b57fa640 1 -- 172.21.15.31:0/1324461583 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f66a8044070 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.073+0000 7f66b57fa640 1 -- 172.21.15.31:0/1324461583 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f66a803d3f0 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.073+0000 7f66be76a640 1 -- 172.21.15.31:0/1324461583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b8108410 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.076+0000 7f66b57fa640 1 -- 172.21.15.31:0/1324461583 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f66a8046450 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.138+0000 7f66be76a640 1 -- 172.21.15.31:0/1324461583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f66b802da20 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.138+0000 7f66b57fa640 1 -- 172.21.15.31:0/1324461583 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f66a8046630 con 0x7f66b8107fb0 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.139+0000 7f66be76a640 1 -- 172.21.15.31:0/1324461583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 msgr2=0x7f66b819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.139+0000 7f66be76a640 1 --2- 172.21.15.31:0/1324461583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b819ac90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f66a80090b0 tx=0x7f66a80046c0 comp rx=0 tx=0).stop 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.139+0000 7f66be76a640 1 -- 172.21.15.31:0/1324461583 shutdown_connections 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.139+0000 7f66be76a640 1 --2- 172.21.15.31:0/1324461583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b8107fb0 0x7f66b819ac90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.139+0000 7f66be76a640 1 -- 172.21.15.31:0/1324461583 >> 172.21.15.31:0/1324461583 conn(0x7f66b8101b90 msgr2=0x7f66b8102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.139+0000 7f66be76a640 1 -- 172.21.15.31:0/1324461583 shutdown_connections 2024-09-16T00:31:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:05.139+0000 7f66be76a640 1 -- 172.21.15.31:0/1324461583 wait complete. 2024-09-16T00:31:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-09-16T00:31:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T00:31:06.856 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:06 smithi031 ceph-mon[30592]: from='client.? 172.21.15.31:0/614898454' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:31:06.856 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:06 smithi031 ceph-mon[30592]: from='client.? 172.21.15.31:0/614898454' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T00:31:06.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:06.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.275+0000 7fbfd5100640 1 Processor -- start 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.275+0000 7fbfd5100640 1 -- start start 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.276+0000 7fbfd5100640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd0107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.276+0000 7fbfd5100640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0107ee0 con 0x7fbfd0107510 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.276+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd0107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.276+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd0107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37722/0 (socket says 172.21.15.31:37722) 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.276+0000 7fbfced9d640 1 -- 172.21.15.31:0/2657855401 learned_addr learned my addr 172.21.15.31:0/2657855401 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.276+0000 7fbfced9d640 1 -- 172.21.15.31:0/2657855401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd01086c0 con 0x7fbfd0107510 2024-09-16T00:31:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.277+0000 7fbfced9d640 1 --2- 172.21.15.31:0/2657855401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd0107910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009820 tx=0x7fbfc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3234a0bb719fe5a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.277+0000 7fbfce59c640 1 -- 172.21.15.31:0/2657855401 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc0037070 con 0x7fbfd0107510 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.277+0000 7fbfce59c640 1 -- 172.21.15.31:0/2657855401 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc0032440 con 0x7fbfd0107510 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.277+0000 7fbfce59c640 1 -- 172.21.15.31:0/2657855401 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc0032740 con 0x7fbfd0107510 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.278+0000 7fbfd5100640 1 -- 172.21.15.31:0/2657855401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 msgr2=0x7fbfd0107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.278+0000 7fbfd5100640 1 --2- 172.21.15.31:0/2657855401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd0107910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009820 tx=0x7fbfc002a270 comp rx=0 tx=0).stop 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.278+0000 7fbfd5100640 1 -- 172.21.15.31:0/2657855401 shutdown_connections 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.278+0000 7fbfd5100640 1 --2- 172.21.15.31:0/2657855401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd0107910 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.278+0000 7fbfd5100640 1 -- 172.21.15.31:0/2657855401 >> 172.21.15.31:0/2657855401 conn(0x7fbfd0102ce0 msgr2=0x7fbfd0105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.278+0000 7fbfd5100640 1 -- 172.21.15.31:0/2657855401 shutdown_connections 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.278+0000 7fbfd5100640 1 -- 172.21.15.31:0/2657855401 wait complete. 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.279+0000 7fbfd5100640 1 Processor -- start 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.279+0000 7fbfd5100640 1 -- start start 2024-09-16T00:31:06.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.279+0000 7fbfd5100640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.279+0000 7fbfd5100640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0108c60 con 0x7fbfd0107510 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.279+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.279+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37738/0 (socket says 172.21.15.31:37738) 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.279+0000 7fbfced9d640 1 -- 172.21.15.31:0/614898454 learned_addr learned my addr 172.21.15.31:0/614898454 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.279+0000 7fbfced9d640 1 -- 172.21.15.31:0/614898454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd019b170 con 0x7fbfd0107510 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.280+0000 7fbfced9d640 1 --2- 172.21.15.31:0/614898454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd019ac30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00090b0 tx=0x7fbfc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.280+0000 7fbfccd99640 1 -- 172.21.15.31:0/614898454 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc0041070 con 0x7fbfd0107510 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.280+0000 7fbfccd99640 1 -- 172.21.15.31:0/614898454 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc003c7a0 con 0x7fbfd0107510 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.280+0000 7fbfccd99640 1 -- 172.21.15.31:0/614898454 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfc0037040 con 0x7fbfd0107510 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.280+0000 7fbfd5100640 1 -- 172.21.15.31:0/614898454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd019b400 con 0x7fbfd0107510 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.280+0000 7fbfd5100640 1 -- 172.21.15.31:0/614898454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd019e0f0 con 0x7fbfd0107510 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.280+0000 7fbfccd99640 1 -- 172.21.15.31:0/614898454 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbfc0044070 con 0x7fbfd0107510 2024-09-16T00:31:06.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.280+0000 7fbfccd99640 1 -- 172.21.15.31:0/614898454 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbfc003d3f0 con 0x7fbfd0107510 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.281+0000 7fbfd5100640 1 -- 172.21.15.31:0/614898454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd01092b0 con 0x7fbfd0107510 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.283+0000 7fbfccd99640 1 -- 172.21.15.31:0/614898454 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbfc0046450 con 0x7fbfd0107510 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.343+0000 7fbfd5100640 1 -- 172.21.15.31:0/614898454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbfd002d980 con 0x7fbfd0107510 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.346+0000 7fbfccd99640 1 -- 172.21.15.31:0/614898454 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fbfc0046630 con 0x7fbfd0107510 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.347+0000 7fbfccd99640 1 -- 172.21.15.31:0/614898454 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbfc0056070 con 0x7fbfd0107510 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.348+0000 7fbfd5100640 1 -- 172.21.15.31:0/614898454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 msgr2=0x7fbfd019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.348+0000 7fbfd5100640 1 --2- 172.21.15.31:0/614898454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd019ac30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00090b0 tx=0x7fbfc00046c0 comp rx=0 tx=0).stop 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.348+0000 7fbfd5100640 1 -- 172.21.15.31:0/614898454 shutdown_connections 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.348+0000 7fbfd5100640 1 --2- 172.21.15.31:0/614898454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0107510 0x7fbfd019ac30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.348+0000 7fbfd5100640 1 -- 172.21.15.31:0/614898454 >> 172.21.15.31:0/614898454 conn(0x7fbfd0102ce0 msgr2=0x7fbfd018ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.348+0000 7fbfd5100640 1 -- 172.21.15.31:0/614898454 shutdown_connections 2024-09-16T00:31:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:06.348+0000 7fbfd5100640 1 -- 172.21.15.31:0/614898454 wait complete. 2024-09-16T00:31:06.871 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-09-16T00:31:07.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:07 smithi031 ceph-mon[30592]: from='client.? 172.21.15.31:0/1966905049' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:07.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.199+0000 7f4f9c5fc640 1 Processor -- start 2024-09-16T00:31:07.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.199+0000 7f4f9c5fc640 1 -- start start 2024-09-16T00:31:07.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.199+0000 7f4f9c5fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f94107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:07.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.199+0000 7f4f9c5fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94107ee0 con 0x7f4f94107510 2024-09-16T00:31:07.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.199+0000 7f4f9a398640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f94107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:07.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.199+0000 7f4f9a398640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f94107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37752/0 (socket says 172.21.15.31:37752) 2024-09-16T00:31:07.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.199+0000 7f4f9a398640 1 -- 172.21.15.31:0/1605736643 learned_addr learned my addr 172.21.15.31:0/1605736643 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:07.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.200+0000 7f4f9a398640 1 -- 172.21.15.31:0/1605736643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f941086c0 con 0x7f4f94107510 2024-09-16T00:31:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.200+0000 7f4f9a398640 1 --2- 172.21.15.31:0/1605736643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f94107910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009820 tx=0x7f4f9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865cdce5dbe3a034 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.200+0000 7f4f99396640 1 -- 172.21.15.31:0/1605736643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f90037070 con 0x7f4f94107510 2024-09-16T00:31:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.201+0000 7f4f99396640 1 -- 172.21.15.31:0/1605736643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f90032900 con 0x7f4f94107510 2024-09-16T00:31:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.201+0000 7f4f99396640 1 -- 172.21.15.31:0/1605736643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f90030850 con 0x7f4f94107510 2024-09-16T00:31:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.201+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1605736643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 msgr2=0x7f4f94107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.201+0000 7f4f9c5fc640 1 --2- 172.21.15.31:0/1605736643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f94107910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009820 tx=0x7f4f9002a270 comp rx=0 tx=0).stop 2024-09-16T00:31:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.201+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1605736643 shutdown_connections 2024-09-16T00:31:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.201+0000 7f4f9c5fc640 1 --2- 172.21.15.31:0/1605736643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f94107910 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:07.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.201+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1605736643 >> 172.21.15.31:0/1605736643 conn(0x7f4f94102ce0 msgr2=0x7f4f94105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.201+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1605736643 shutdown_connections 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.201+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1605736643 wait complete. 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.202+0000 7f4f9c5fc640 1 Processor -- start 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.202+0000 7f4f9c5fc640 1 -- start start 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.202+0000 7f4f9c5fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f9419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.202+0000 7f4f9c5fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9410ba70 con 0x7f4f94107510 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.202+0000 7f4f9a398640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f9419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.202+0000 7f4f9a398640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f9419b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37756/0 (socket says 172.21.15.31:37756) 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.202+0000 7f4f9a398640 1 -- 172.21.15.31:0/1966905049 learned_addr learned my addr 172.21.15.31:0/1966905049 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.202+0000 7f4f9a398640 1 -- 172.21.15.31:0/1966905049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9419ba20 con 0x7f4f94107510 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.203+0000 7f4f9a398640 1 --2- 172.21.15.31:0/1966905049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f9419b4e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009940 tx=0x7f4f90032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.203+0000 7f4f8b7fe640 1 -- 172.21.15.31:0/1966905049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f90042070 con 0x7f4f94107510 2024-09-16T00:31:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.203+0000 7f4f8b7fe640 1 -- 172.21.15.31:0/1966905049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f900308c0 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.203+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1966905049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9419bcb0 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.203+0000 7f4f8b7fe640 1 -- 172.21.15.31:0/1966905049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f90037040 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.203+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1966905049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9419e9a0 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.203+0000 7f4f8b7fe640 1 -- 172.21.15.31:0/1966905049 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4f90045070 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.203+0000 7f4f8b7fe640 1 -- 172.21.15.31:0/1966905049 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4f90047bc0 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.204+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1966905049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f9410bbf0 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.206+0000 7f4f8b7fe640 1 -- 172.21.15.31:0/1966905049 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4f900327b0 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.266+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1966905049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4f9419ee80 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.266+0000 7f4f8b7fe640 1 -- 172.21.15.31:0/1966905049 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f4f9003c870 con 0x7f4f94107510 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.267+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1966905049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 msgr2=0x7f4f9419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.267+0000 7f4f9c5fc640 1 --2- 172.21.15.31:0/1966905049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f9419b4e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009940 tx=0x7f4f90032ce0 comp rx=0 tx=0).stop 2024-09-16T00:31:07.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.267+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1966905049 shutdown_connections 2024-09-16T00:31:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.267+0000 7f4f9c5fc640 1 --2- 172.21.15.31:0/1966905049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f94107510 0x7f4f9419b4e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.267+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1966905049 >> 172.21.15.31:0/1966905049 conn(0x7f4f94102ce0 msgr2=0x7f4f94103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.267+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1966905049 shutdown_connections 2024-09-16T00:31:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:07.267+0000 7f4f9c5fc640 1 -- 172.21.15.31:0/1966905049 wait complete. 2024-09-16T00:31:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-09-16T00:31:08.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:07 smithi031 systemd[1]: Stopping Ceph mon.a for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:31:08.592 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a[30588]: 2024-09-16T00:31:08.150+0000 7f439e0da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:08.592 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a[30588]: 2024-09-16T00:31:08.150+0000 7f439e0da640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T00:31:08.592 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:08 smithi031 podman[30834]: 2024-09-16 00:31:08.313955997 +0000 UTC m=+0.274663089 container died 0c56ea5db5bcb8a0d5236e7c933bac11cb336e71c4779be81ab8cd7708baf0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-16T00:31:08.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:08 smithi031 podman[30834]: 2024-09-16 00:31:08.699853982 +0000 UTC m=+0.660561078 container cleanup 0c56ea5db5bcb8a0d5236e7c933bac11cb336e71c4779be81ab8cd7708baf0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T00:31:08.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:08 smithi031 bash[30834]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a 2024-09-16T00:31:09.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:08 smithi031 podman[30846]: 2024-09-16 00:31:08.909189921 +0000 UTC m=+0.593359262 container remove 0c56ea5db5bcb8a0d5236e7c933bac11cb336e71c4779be81ab8cd7708baf0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:31:09.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 systemd[1]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T00:31:09.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 systemd[1]: Stopped Ceph mon.a for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:31:09.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 systemd[1]: Starting Ceph mon.a for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:31:09.479 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 podman[30937]: 2024-09-16 00:31:09.218440944 +0000 UTC m=+0.086780173 container create 4b873356c0c186fc79a2931a7147f1b8aa118f993a84e7574bf8e5b6740d9d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:31:09.479 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 podman[30937]: 2024-09-16 00:31:09.14833217 +0000 UTC m=+0.016671366 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:31:09.479 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 podman[30937]: 2024-09-16 00:31:09.454314756 +0000 UTC m=+0.322653962 container init 4b873356c0c186fc79a2931a7147f1b8aa118f993a84e7574bf8e5b6740d9d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:09.479 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 podman[30937]: 2024-09-16 00:31:09.457466334 +0000 UTC m=+0.325805530 container start 4b873356c0c186fc79a2931a7147f1b8aa118f993a84e7574bf8e5b6740d9d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T00:31:09.550 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T00:31:09.739 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T00:31:09.739 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T00:31:09.739 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: pidfile_write: ignore empty --pid-file 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: load: jerasure load: lrc 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: RocksDB version: 7.9.2 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Git sha 0 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: DB SUMMARY 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: DB Session ID: L4DAM50FEDS1K6GL79YO 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T00:31:09.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104226 ; 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.env: 0x562da1bc6940 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.info_log: 0x562da371d6e0 2024-09-16T00:31:09.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:31:09.742 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.statistics: (nil) 2024-09-16T00:31:09.742 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.use_fsync: 0 2024-09-16T00:31:09.742 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:31:09.742 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:31:09.742 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:31:09.742 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:31:09.742 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:31:09.742 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:31:09.742 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.db_log_dir: 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.wal_dir: 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:31:09.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.write_buffer_manager: 0x562da372cdc0 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:31:09.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:31:09.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.unordered_write: 0 2024-09-16T00:31:09.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.row_cache: None 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.wal_filter: None 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.wal_compression: 0 2024-09-16T00:31:09.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:31:09.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:31:09.748 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T00:31:09.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:31:09.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:31:09.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:31:09.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:31:09.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:31:09.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_open_files: -1 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Compression algorithms supported: 2024-09-16T00:31:09.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: kZSTD supported: 0 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.merge_operator: 2024-09-16T00:31:09.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_filter: None 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562da371d320) 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_type: 0 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_index_type: 0 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_shortening: 1 2024-09-16T00:31:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: checksum: 4 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: no_block_cache: 0 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache: 0x562da37111f0 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_options: 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: capacity : 536870912 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_shard_bits : 4 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: strict_capacity_limit : 0 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_compressed: (nil) 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: persistent_cache: (nil) 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size: 4096 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size_deviation: 10 2024-09-16T00:31:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_restart_interval: 16 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_block_restart_interval: 1 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: metadata_block_size: 4096 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: partition_filters: 0 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: use_delta_encoding: 1 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: filter_policy: bloomfilter 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: whole_key_filtering: 1 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: verify_compression: 0 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: format_version: 5 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_index_compression: 1 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_align: 0 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: max_auto_readahead_size: 262144 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: prepopulate_block_cache: 0 2024-09-16T00:31:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout: initial_auto_readahead_size: 8192 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression: NoCompression 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.num_levels: 7 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:31:09.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:31:09.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:31:09.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:31:09.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:31:09.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.ttl: 2592000 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:31:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 41594f99-3ba9-4ab9-a084-39287e02eb73 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446669485052, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446669486601, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99451, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97345, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14048, "raw_average_key_size": 49, "raw_value_size": 89673, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726446669, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "41594f99-3ba9-4ab9-a084-39287e02eb73", "db_session_id": "L4DAM50FEDS1K6GL79YO", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446669486737, "job": 1, "event": "recovery_finished"} 2024-09-16T00:31:09.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562da373ee00 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: DB pointer 0x562da383a000 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** DB Stats ** 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:31:09.762 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: L0 2/0 98.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: Sum 2/0 98.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 102.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T00:31:09.763 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative compaction: 0.00 GB write, 20.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval compaction: 0.00 GB write, 20.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache BinnedLRUCache@0x562da37111f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-16T00:31:09.764 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-16T00:31:09.765 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:31:09.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: starting mon.a rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:09.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mon.a@-1(???) e1 preinit fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:09.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mon.a@-1(???).mds e1 new map 2024-09-16T00:31:09.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mon.a@-1(???).mds e1 print_map 2024-09-16T00:31:09.765 INFO:journalctl@ceph.mon.a.smithi031.stdout: e1 2024-09-16T00:31:09.765 INFO:journalctl@ceph.mon.a.smithi031.stdout: btime 2024-09-16T00:31:04:619302+0000 2024-09-16T00:31:09.765 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:31:09.765 INFO:journalctl@ceph.mon.a.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout: legacy client fscid: -1 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout: No filesystems configured 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 bash[30937]: 4b873356c0c186fc79a2931a7147f1b8aa118f993a84e7574bf8e5b6740d9d2e 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 systemd[1]: Started Ceph mon.a for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: monmap epoch 1 2024-09-16T00:31:09.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:09.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: last_changed 2024-09-16T00:31:01.292640+0000 2024-09-16T00:31:09.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:31:09.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: min_mon_release 19 (squid) 2024-09-16T00:31:09.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: election_strategy: 1 2024-09-16T00:31:09.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:31:09.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: fsmap 2024-09-16T00:31:09.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T00:31:09.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:09 smithi031 ceph-mon[30956]: mgrmap e1: no daemons active 2024-09-16T00:31:10.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.949+0000 7facf4f9c640 1 Processor -- start 2024-09-16T00:31:10.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.950+0000 7facf4f9c640 1 -- start start 2024-09-16T00:31:10.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.950+0000 7facf4f9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf010a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.950+0000 7facf4f9c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7facf010a880 con 0x7facf0107f50 2024-09-16T00:31:10.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.950+0000 7facee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf010a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.950+0000 7facee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf010a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56958/0 (socket says 172.21.15.31:56958) 2024-09-16T00:31:10.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.950+0000 7facee59c640 1 -- 172.21.15.31:0/3233497015 learned_addr learned my addr 172.21.15.31:0/3233497015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:10.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.950+0000 7facee59c640 1 -- 172.21.15.31:0/3233497015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf010aa00 con 0x7facf0107f50 2024-09-16T00:31:10.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.952+0000 7facee59c640 1 --2- 172.21.15.31:0/3233497015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf010a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7face0009820 tx=0x7face002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2235652cb635aac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:10.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.952+0000 7facedd9b640 1 -- 172.21.15.31:0/3233497015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face0037070 con 0x7facf0107f50 2024-09-16T00:31:10.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.952+0000 7facedd9b640 1 -- 172.21.15.31:0/3233497015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7face0032900 con 0x7facf0107f50 2024-09-16T00:31:10.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.952+0000 7facedd9b640 1 -- 172.21.15.31:0/3233497015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face0030850 con 0x7facf0107f50 2024-09-16T00:31:10.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.953+0000 7facf4f9c640 1 -- 172.21.15.31:0/3233497015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 msgr2=0x7facf010a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.953+0000 7facf4f9c640 1 --2- 172.21.15.31:0/3233497015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf010a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7face0009820 tx=0x7face002a270 comp rx=0 tx=0).stop 2024-09-16T00:31:10.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.953+0000 7facf4f9c640 1 -- 172.21.15.31:0/3233497015 shutdown_connections 2024-09-16T00:31:10.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.953+0000 7facf4f9c640 1 --2- 172.21.15.31:0/3233497015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf010a340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.953+0000 7facf4f9c640 1 -- 172.21.15.31:0/3233497015 >> 172.21.15.31:0/3233497015 conn(0x7facf0101af0 msgr2=0x7facf0103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.953+0000 7facf4f9c640 1 -- 172.21.15.31:0/3233497015 shutdown_connections 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.953+0000 7facf4f9c640 1 -- 172.21.15.31:0/3233497015 wait complete. 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.954+0000 7facf4f9c640 1 Processor -- start 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.954+0000 7facf4f9c640 1 -- start start 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.954+0000 7facf4f9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf01077d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.954+0000 7facf4f9c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7facf010b9e0 con 0x7facf0107f50 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.954+0000 7facee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf01077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.954+0000 7facee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf01077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56972/0 (socket says 172.21.15.31:56972) 2024-09-16T00:31:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.954+0000 7facee59c640 1 -- 172.21.15.31:0/3295041207 learned_addr learned my addr 172.21.15.31:0/3295041207 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.954+0000 7facee59c640 1 -- 172.21.15.31:0/3295041207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf0107d10 con 0x7facf0107f50 2024-09-16T00:31:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.955+0000 7facee59c640 1 --2- 172.21.15.31:0/3295041207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf01077d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7face0032ce0 tx=0x7face0032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.955+0000 7face77fe640 1 -- 172.21.15.31:0/3295041207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face0042070 con 0x7facf0107f50 2024-09-16T00:31:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.955+0000 7face77fe640 1 -- 172.21.15.31:0/3295041207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7face00308c0 con 0x7facf0107f50 2024-09-16T00:31:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.955+0000 7facf4f9c640 1 -- 172.21.15.31:0/3295041207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf0105f00 con 0x7facf0107f50 2024-09-16T00:31:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.955+0000 7face77fe640 1 -- 172.21.15.31:0/3295041207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face0037040 con 0x7facf0107f50 2024-09-16T00:31:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.955+0000 7facf4f9c640 1 -- 172.21.15.31:0/3295041207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf0106440 con 0x7facf0107f50 2024-09-16T00:31:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.955+0000 7face77fe640 1 -- 172.21.15.31:0/3295041207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7face0045070 con 0x7facf0107f50 2024-09-16T00:31:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.955+0000 7face77fe640 1 -- 172.21.15.31:0/3295041207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7face0047bc0 con 0x7facf0107f50 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.956+0000 7facf4f9c640 1 -- 172.21.15.31:0/3295041207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf01083b0 con 0x7facf0107f50 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:09.958+0000 7face77fe640 1 -- 172.21.15.31:0/3295041207 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7face003c830 con 0x7facf0107f50 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:10.017+0000 7facf4f9c640 1 -- 172.21.15.31:0/3295041207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7facf01067b0 con 0x7facf0107f50 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:10.021+0000 7face77fe640 1 -- 172.21.15.31:0/3295041207 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7face003ca10 con 0x7facf0107f50 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:10.022+0000 7facf4f9c640 1 -- 172.21.15.31:0/3295041207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 msgr2=0x7facf01077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:10.022+0000 7facf4f9c640 1 --2- 172.21.15.31:0/3295041207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf01077d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7face0032ce0 tx=0x7face0032d10 comp rx=0 tx=0).stop 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:10.022+0000 7facf4f9c640 1 -- 172.21.15.31:0/3295041207 shutdown_connections 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:10.022+0000 7facf4f9c640 1 --2- 172.21.15.31:0/3295041207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf0107f50 0x7facf01077d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:10.022+0000 7facf4f9c640 1 -- 172.21.15.31:0/3295041207 >> 172.21.15.31:0/3295041207 conn(0x7facf0101af0 msgr2=0x7facf01026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:10.022+0000 7facf4f9c640 1 -- 172.21.15.31:0/3295041207 shutdown_connections 2024-09-16T00:31:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:10.022+0000 7facf4f9c640 1 -- 172.21.15.31:0/3295041207 wait complete. 2024-09-16T00:31:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T00:31:10.586 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:31:10.587 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-09-16T00:31:10.587 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T00:31:10.587 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T00:31:10.863 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.x 2024-09-16T00:31:10.863 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-16T00:31:11.017 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968.target.wants/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@.service. 2024-09-16T00:31:11.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:11 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3295041207' entity='client.admin' 2024-09-16T00:31:11.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 systemd[1]: Starting Ceph mgr.x for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:31:11.550 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 podman[31172]: 2024-09-16 00:31:11.165704063 +0000 UTC m=+0.016302104 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:31:11.550 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 podman[31172]: 2024-09-16 00:31:11.264234974 +0000 UTC m=+0.114833014 container create 509e457288037a18e446ff7e038bab5ec60bd6b96146c4698775424a52223407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:11.550 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 podman[31172]: 2024-09-16 00:31:11.475458449 +0000 UTC m=+0.326056495 container init 509e457288037a18e446ff7e038bab5ec60bd6b96146c4698775424a52223407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T00:31:11.550 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 podman[31172]: 2024-09-16 00:31:11.47861084 +0000 UTC m=+0.329208880 container start 509e457288037a18e446ff7e038bab5ec60bd6b96146c4698775424a52223407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T00:31:11.550 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 bash[31172]: 509e457288037a18e446ff7e038bab5ec60bd6b96146c4698775424a52223407 2024-09-16T00:31:11.564 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:31:11.564 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-16T00:31:11.572 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:31:11.572 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T00:31:11.572 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-16T00:31:11.572 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:31:11.582 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:31:11.582 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-16T00:31:11.590 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:31:11.591 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T00:31:11.591 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-16T00:31:11.591 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T00:31:11.591 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-09-16T00:31:11.591 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-09-16T00:31:11.832 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 systemd[1]: Started Ceph mgr.x for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:31:11.832 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:11.609+0000 7f578afc7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:31:11.832 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:11.670+0000 7f578afc7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:31:11.832 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:11.832+0000 7f578afc7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:31:12.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:12 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3582507622' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:31:12.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:11 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:11.903+0000 7f578afc7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:31:12.126 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:12.064+0000 7f578afc7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:31:12.551 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:12.213+0000 7f578afc7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:31:12.668 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "ccb601ea-73c2-11ef-bceb-c7b262605968", 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:31:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:31:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:31:12.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:31:12.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:31:12.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:31:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:31:04:619302+0000", 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:31:12.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:31:04.620011+0000", 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.958+0000 7fcf1e5ad640 1 Processor -- start 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.959+0000 7fcf1e5ad640 1 -- start start 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.959+0000 7fcf1e5ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf18107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.959+0000 7fcf1e5ad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18105fc0 con 0x7fcf18107a80 2024-09-16T00:31:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.959+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf18107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.959+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf18107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56988/0 (socket says 172.21.15.31:56988) 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.959+0000 7fcf17fff640 1 -- 172.21.15.31:0/123416045 learned_addr learned my addr 172.21.15.31:0/123416045 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.960+0000 7fcf17fff640 1 -- 172.21.15.31:0/123416045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf18106140 con 0x7fcf18107a80 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.960+0000 7fcf17fff640 1 --2- 172.21.15.31:0/123416045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf18107e80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800bac0 tx=0x7fcf0802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b1dbe89cf34dba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.961+0000 7fcf16ffd640 1 -- 172.21.15.31:0/123416045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf080326d0 con 0x7fcf18107a80 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.961+0000 7fcf16ffd640 1 -- 172.21.15.31:0/123416045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcf08038070 con 0x7fcf18107a80 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.961+0000 7fcf16ffd640 1 -- 172.21.15.31:0/123416045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf08033d70 con 0x7fcf18107a80 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.961+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/123416045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 msgr2=0x7fcf18107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.961+0000 7fcf1e5ad640 1 --2- 172.21.15.31:0/123416045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf18107e80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800bac0 tx=0x7fcf0802c920 comp rx=0 tx=0).stop 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.961+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/123416045 shutdown_connections 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.961+0000 7fcf1e5ad640 1 --2- 172.21.15.31:0/123416045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf18107e80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.961+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/123416045 >> 172.21.15.31:0/123416045 conn(0x7fcf18101af0 msgr2=0x7fcf18103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.961+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/123416045 shutdown_connections 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.962+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/123416045 wait complete. 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.962+0000 7fcf1e5ad640 1 Processor -- start 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.962+0000 7fcf1e5ad640 1 -- start start 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.962+0000 7fcf1e5ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf1819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.962+0000 7fcf1e5ad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18074b20 con 0x7fcf18107a80 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf1819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf1819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56998/0 (socket says 172.21.15.31:56998) 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf17fff640 1 -- 172.21.15.31:0/3582507622 learned_addr learned my addr 172.21.15.31:0/3582507622 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf17fff640 1 -- 172.21.15.31:0/3582507622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1819dd70 con 0x7fcf18107a80 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf17fff640 1 --2- 172.21.15.31:0/3582507622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf1819d830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800b2c0 tx=0x7fcf080328b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf157fa640 1 -- 172.21.15.31:0/3582507622 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf08042070 con 0x7fcf18107a80 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf157fa640 1 -- 172.21.15.31:0/3582507622 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcf0803e900 con 0x7fcf18107a80 2024-09-16T00:31:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/3582507622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1819e000 con 0x7fcf18107a80 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf157fa640 1 -- 172.21.15.31:0/3582507622 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf0803c970 con 0x7fcf18107a80 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.963+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/3582507622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf181a0cf0 con 0x7fcf18107a80 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.964+0000 7fcf157fa640 1 -- 172.21.15.31:0/3582507622 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf08045070 con 0x7fcf18107a80 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.964+0000 7fcf157fa640 1 -- 172.21.15.31:0/3582507622 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcf08030050 con 0x7fcf18107a80 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.964+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/3582507622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf18107e80 con 0x7fcf18107a80 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:11.966+0000 7fcf157fa640 1 -- 172.21.15.31:0/3582507622 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fcf0803eab0 con 0x7fcf18107a80 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:12.029+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/3582507622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcf181a1240 con 0x7fcf18107a80 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:12.029+0000 7fcf157fa640 1 -- 172.21.15.31:0/3582507622 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fcf0803ec90 con 0x7fcf18107a80 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:12.030+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/3582507622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 msgr2=0x7fcf1819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:12.030+0000 7fcf1e5ad640 1 --2- 172.21.15.31:0/3582507622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf1819d830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800b2c0 tx=0x7fcf080328b0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:12.030+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/3582507622 shutdown_connections 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:12.030+0000 7fcf1e5ad640 1 --2- 172.21.15.31:0/3582507622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf18107a80 0x7fcf1819d830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:12.030+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/3582507622 >> 172.21.15.31:0/3582507622 conn(0x7fcf18101af0 msgr2=0x7fcf1818f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.687 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:12.030+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/3582507622 shutdown_connections 2024-09-16T00:31:12.687 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:12.030+0000 7fcf1e5ad640 1 -- 172.21.15.31:0/3582507622 wait complete. 2024-09-16T00:31:12.687 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-09-16T00:31:12.810 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:12.551+0000 7f578afc7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:31:12.810 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:12.614+0000 7f578afc7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:31:12.810 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:12.810+0000 7f578afc7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:31:14.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:13 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:13.734+0000 7f578afc7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:31:14.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:13 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:13.804+0000 7f578afc7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:31:14.851 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:14 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:14.566+0000 7f578afc7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:31:14.851 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:14 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:14.687+0000 7f578afc7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:31:15.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:14 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:14.850+0000 7f578afc7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:31:15.126 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:14 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:14.917+0000 7f578afc7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:31:15.126 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:15.077+0000 7f578afc7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:31:15.456 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/564573315' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:31:15.457 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:15.152+0000 7f578afc7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:31:15.457 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:15.219+0000 7f578afc7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:31:15.457 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:15.388+0000 7f578afc7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:31:15.457 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:15.456+0000 7f578afc7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:31:15.741 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:15.529+0000 7f578afc7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:31:15.741 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:15.616+0000 7f578afc7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:31:15.741 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:15.679+0000 7f578afc7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:31:15.741 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:15 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:15.741+0000 7f578afc7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:31:15.809 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "ccb601ea-73c2-11ef-bceb-c7b262605968", 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:31:15.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:31:15.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:31:15.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:31:04:619302+0000", 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:31:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:31:04.620011+0000", 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.080+0000 7fae1f69a640 1 Processor -- start 2024-09-16T00:31:15.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.080+0000 7fae1f69a640 1 -- start start 2024-09-16T00:31:15.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.080+0000 7fae1f69a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae18107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.080+0000 7fae1f69a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fae18107ee0 con 0x7fae18107510 2024-09-16T00:31:15.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.080+0000 7fae1d436640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae18107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.080+0000 7fae1d436640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae18107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57010/0 (socket says 172.21.15.31:57010) 2024-09-16T00:31:15.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.080+0000 7fae1d436640 1 -- 172.21.15.31:0/2020292622 learned_addr learned my addr 172.21.15.31:0/2020292622 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:15.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.081+0000 7fae1d436640 1 -- 172.21.15.31:0/2020292622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae181086c0 con 0x7fae18107510 2024-09-16T00:31:15.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.081+0000 7fae1d436640 1 --2- 172.21.15.31:0/2020292622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae18107910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae14009820 tx=0x7fae1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7cc65875940153 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae0ffff640 1 -- 172.21.15.31:0/2020292622 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae14037070 con 0x7fae18107510 2024-09-16T00:31:15.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae0ffff640 1 -- 172.21.15.31:0/2020292622 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fae14032900 con 0x7fae18107510 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae0ffff640 1 -- 172.21.15.31:0/2020292622 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae14030850 con 0x7fae18107510 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae1f69a640 1 -- 172.21.15.31:0/2020292622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 msgr2=0x7fae18107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae1f69a640 1 --2- 172.21.15.31:0/2020292622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae18107910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae14009820 tx=0x7fae1402a270 comp rx=0 tx=0).stop 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae1f69a640 1 -- 172.21.15.31:0/2020292622 shutdown_connections 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae1f69a640 1 --2- 172.21.15.31:0/2020292622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae18107910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae1f69a640 1 -- 172.21.15.31:0/2020292622 >> 172.21.15.31:0/2020292622 conn(0x7fae18102ce0 msgr2=0x7fae18105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae1f69a640 1 -- 172.21.15.31:0/2020292622 shutdown_connections 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.082+0000 7fae1f69a640 1 -- 172.21.15.31:0/2020292622 wait complete. 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.083+0000 7fae1f69a640 1 Processor -- start 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.083+0000 7fae1f69a640 1 -- start start 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae1f69a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae1819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae1f69a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fae1810ba70 con 0x7fae18107510 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae1d436640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae1819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae1d436640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae1819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57012/0 (socket says 172.21.15.31:57012) 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae1d436640 1 -- 172.21.15.31:0/564573315 learned_addr learned my addr 172.21.15.31:0/564573315 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae1d436640 1 -- 172.21.15.31:0/564573315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1819ba20 con 0x7fae18107510 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae1d436640 1 --2- 172.21.15.31:0/564573315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae1819b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae14009940 tx=0x7fae14032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae0e7fc640 1 -- 172.21.15.31:0/564573315 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae14042070 con 0x7fae18107510 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae0e7fc640 1 -- 172.21.15.31:0/564573315 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fae140308c0 con 0x7fae18107510 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae0e7fc640 1 -- 172.21.15.31:0/564573315 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae14037040 con 0x7fae18107510 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae1f69a640 1 -- 172.21.15.31:0/564573315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1819bcb0 con 0x7fae18107510 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.084+0000 7fae1f69a640 1 -- 172.21.15.31:0/564573315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1819e9a0 con 0x7fae18107510 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.085+0000 7fae0e7fc640 1 -- 172.21.15.31:0/564573315 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fae14045070 con 0x7fae18107510 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.085+0000 7fae0e7fc640 1 -- 172.21.15.31:0/564573315 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fae14047bb0 con 0x7fae18107510 2024-09-16T00:31:15.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.085+0000 7fae1f69a640 1 -- 172.21.15.31:0/564573315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae1810bbf0 con 0x7fae18107510 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.087+0000 7fae0e7fc640 1 -- 172.21.15.31:0/564573315 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fae140327b0 con 0x7fae18107510 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.150+0000 7fae1f69a640 1 -- 172.21.15.31:0/564573315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fae1819eb70 con 0x7fae18107510 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.151+0000 7fae0e7fc640 1 -- 172.21.15.31:0/564573315 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fae1403c760 con 0x7fae18107510 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.152+0000 7fae1f69a640 1 -- 172.21.15.31:0/564573315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 msgr2=0x7fae1819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.152+0000 7fae1f69a640 1 --2- 172.21.15.31:0/564573315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae1819b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae14009940 tx=0x7fae14032ce0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.152+0000 7fae1f69a640 1 -- 172.21.15.31:0/564573315 shutdown_connections 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.152+0000 7fae1f69a640 1 --2- 172.21.15.31:0/564573315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae18107510 0x7fae1819b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.152+0000 7fae1f69a640 1 -- 172.21.15.31:0/564573315 >> 172.21.15.31:0/564573315 conn(0x7fae18102ce0 msgr2=0x7fae18103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.152+0000 7fae1f69a640 1 -- 172.21.15.31:0/564573315 shutdown_connections 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:15.152+0000 7fae1f69a640 1 -- 172.21.15.31:0/564573315 wait complete. 2024-09-16T00:31:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2024-09-16T00:31:16.875 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:16.573+0000 7f578afc7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: Activating manager daemon x 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: mgrmap e2: x(active, starting, since 0.0029006s) 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: Manager daemon x is now available 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' 2024-09-16T00:31:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' 2024-09-16T00:31:16.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:16 smithi031 ceph-mon[30956]: from='mgr.14100 172.21.15.31:0/3934789363' entity='mgr.x' 2024-09-16T00:31:18.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:18 smithi031 ceph-mon[30956]: mgrmap e3: x(active, since 1.00722s) 2024-09-16T00:31:18.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:18 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2099568471' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:31:19.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:19.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:31:19.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "ccb601ea-73c2-11ef-bceb-c7b262605968", 2024-09-16T00:31:19.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:31:19.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:31:19.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:31:19.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:19.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:31:19.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:19.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:31:19.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:31:19.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:31:19.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:31:19.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:31:04:619302+0000", 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:31:19.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:31:04.620011+0000", 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.227+0000 7fcccf112640 1 Processor -- start 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.227+0000 7fcccf112640 1 -- start start 2024-09-16T00:31:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.227+0000 7fcccf112640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc8107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.227+0000 7fcccf112640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8107ee0 con 0x7fccc8107510 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.228+0000 7fcccceae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc8107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.228+0000 7fcccceae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc8107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42160/0 (socket says 172.21.15.31:42160) 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.228+0000 7fcccceae640 1 -- 172.21.15.31:0/2057295167 learned_addr learned my addr 172.21.15.31:0/2057295167 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.228+0000 7fcccceae640 1 -- 172.21.15.31:0/2057295167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc81086c0 con 0x7fccc8107510 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.229+0000 7fcccceae640 1 --2- 172.21.15.31:0/2057295167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc8107910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009820 tx=0x7fccb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fef463f2c1d2f44 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.229+0000 7fccc77fe640 1 -- 172.21.15.31:0/2057295167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccb8037070 con 0x7fccc8107510 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.229+0000 7fccc77fe640 1 -- 172.21.15.31:0/2057295167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fccb8032900 con 0x7fccc8107510 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.229+0000 7fccc77fe640 1 -- 172.21.15.31:0/2057295167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccb8030850 con 0x7fccc8107510 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.230+0000 7fcccf112640 1 -- 172.21.15.31:0/2057295167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 msgr2=0x7fccc8107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.230+0000 7fcccf112640 1 --2- 172.21.15.31:0/2057295167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc8107910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009820 tx=0x7fccb802a270 comp rx=0 tx=0).stop 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.230+0000 7fcccf112640 1 -- 172.21.15.31:0/2057295167 shutdown_connections 2024-09-16T00:31:19.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.230+0000 7fcccf112640 1 --2- 172.21.15.31:0/2057295167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc8107910 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.230+0000 7fcccf112640 1 -- 172.21.15.31:0/2057295167 >> 172.21.15.31:0/2057295167 conn(0x7fccc8102ce0 msgr2=0x7fccc8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.230+0000 7fcccf112640 1 -- 172.21.15.31:0/2057295167 shutdown_connections 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.230+0000 7fcccf112640 1 -- 172.21.15.31:0/2057295167 wait complete. 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.230+0000 7fcccf112640 1 Processor -- start 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.230+0000 7fcccf112640 1 -- start start 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fcccf112640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fcccf112640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810ba70 con 0x7fccc8107510 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fcccceae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fcccceae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42170/0 (socket says 172.21.15.31:42170) 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fcccceae640 1 -- 172.21.15.31:0/2099568471 learned_addr learned my addr 172.21.15.31:0/2099568471 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fcccceae640 1 -- 172.21.15.31:0/2099568471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc819ba20 con 0x7fccc8107510 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fcccceae640 1 --2- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc819b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fccb80097f0 tx=0x7fccb8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fccc5ffb640 1 -- 172.21.15.31:0/2099568471 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccb8037040 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fccc5ffb640 1 -- 172.21.15.31:0/2099568471 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fccb8040070 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fccc5ffb640 1 -- 172.21.15.31:0/2099568471 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccb803a390 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.231+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc819bcb0 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.232+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc819e9a0 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.232+0000 7fccc5ffb640 1 -- 172.21.15.31:0/2099568471 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7fccb803b430 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.233+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc810bbf0 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.233+0000 7fccc5ffb640 1 --2- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7fccb00390a0 0x7fccb003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.233+0000 7fccc5ffb640 1 -- 172.21.15.31:0/2099568471 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fccb806c160 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.233+0000 7fccc7fff640 1 --2- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7fccb00390a0 0x7fccb003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.233+0000 7fccc7fff640 1 --2- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7fccb00390a0 0x7fccb003b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0098c0 tx=0x7fccbc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.238+0000 7fccc5ffb640 1 -- 172.21.15.31:0/2099568471 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccb8032900 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.484+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fccc819eb70 con 0x7fccc8107510 2024-09-16T00:31:19.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.485+0000 7fccc5ffb640 1 -- 172.21.15.31:0/2099568471 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fccb8043c30 con 0x7fccc8107510 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7fccb00390a0 msgr2=0x7fccb003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 --2- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7fccb00390a0 0x7fccb003b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0098c0 tx=0x7fccbc006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 msgr2=0x7fccc819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 --2- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc819b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fccb80097f0 tx=0x7fccb8030d80 comp rx=0 tx=0).stop 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 shutdown_connections 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 --2- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7fccb00390a0 0x7fccb003b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 --2- 172.21.15.31:0/2099568471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc8107510 0x7fccc819b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 >> 172.21.15.31:0/2099568471 conn(0x7fccc8102ce0 msgr2=0x7fccc8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 shutdown_connections 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:18.487+0000 7fcccf112640 1 -- 172.21.15.31:0/2099568471 wait complete. 2024-09-16T00:31:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-09-16T00:31:19.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:19 smithi031 ceph-mon[30956]: mgrmap e4: x(active, since 2s) 2024-09-16T00:31:20.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:20 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2143513698' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:31:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:31:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-16T00:31:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:31:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:31:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:31:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.451+0000 7f857d045640 1 Processor -- start 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.451+0000 7f857d045640 1 -- start start 2024-09-16T00:31:25.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.451+0000 7f857d045640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f85781078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.451+0000 7f857d045640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8578107e90 con 0x7f85781074c0 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.452+0000 7f8576d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f85781078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.452+0000 7f8576d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f85781078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42176/0 (socket says 172.21.15.31:42176) 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.452+0000 7f8576d9d640 1 -- 172.21.15.31:0/252226950 learned_addr learned my addr 172.21.15.31:0/252226950 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.452+0000 7f8576d9d640 1 -- 172.21.15.31:0/252226950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8578108670 con 0x7f85781074c0 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.452+0000 7f8576d9d640 1 --2- 172.21.15.31:0/252226950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f85781078c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8568009820 tx=0x7f856802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f48bc884382c7e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f8575d9b640 1 -- 172.21.15.31:0/252226950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8568037070 con 0x7f85781074c0 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f8575d9b640 1 -- 172.21.15.31:0/252226950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8568032900 con 0x7f85781074c0 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f8575d9b640 1 -- 172.21.15.31:0/252226950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8568030850 con 0x7f85781074c0 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f857d045640 1 -- 172.21.15.31:0/252226950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 msgr2=0x7f85781078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:25.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f857d045640 1 --2- 172.21.15.31:0/252226950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f85781078c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8568009820 tx=0x7f856802a270 comp rx=0 tx=0).stop 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f857d045640 1 -- 172.21.15.31:0/252226950 shutdown_connections 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f857d045640 1 --2- 172.21.15.31:0/252226950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f85781078c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f857d045640 1 -- 172.21.15.31:0/252226950 >> 172.21.15.31:0/252226950 conn(0x7f8578102c10 msgr2=0x7f8578105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f857d045640 1 -- 172.21.15.31:0/252226950 shutdown_connections 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.453+0000 7f857d045640 1 -- 172.21.15.31:0/252226950 wait complete. 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.454+0000 7f857d045640 1 Processor -- start 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.454+0000 7f857d045640 1 -- start start 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.454+0000 7f857d045640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f857807ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.454+0000 7f857d045640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f857810ba20 con 0x7f85781074c0 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f8576d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f857807ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f8576d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f857807ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42184/0 (socket says 172.21.15.31:42184) 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f8576d9d640 1 -- 172.21.15.31:0/2143513698 learned_addr learned my addr 172.21.15.31:0/2143513698 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:25.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f8576d9d640 1 -- 172.21.15.31:0/2143513698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857807cf70 con 0x7f85781074c0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f8576d9d640 1 --2- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f857807ca30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8568009940 tx=0x7f8568032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f855ffff640 1 -- 172.21.15.31:0/2143513698 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8568042070 con 0x7f85781074c0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f855ffff640 1 -- 172.21.15.31:0/2143513698 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85680308c0 con 0x7f85781074c0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857807b160 con 0x7f85781074c0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f855ffff640 1 -- 172.21.15.31:0/2143513698 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8568037040 con 0x7f85781074c0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.455+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857807b6a0 con 0x7f85781074c0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.456+0000 7f855ffff640 1 -- 172.21.15.31:0/2143513698 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f8568045070 con 0x7f85781074c0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.456+0000 7f855ffff640 1 --2- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f85600391b0 0x7f856003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.456+0000 7f855ffff640 1 -- 172.21.15.31:0/2143513698 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f856806c1d0 con 0x7f85781074c0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.456+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8578107940 con 0x7f85781074c0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.456+0000 7f857659c640 1 --2- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f85600391b0 0x7f856003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.457+0000 7f857659c640 1 --2- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f85600391b0 0x7f856003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f856c0098c0 tx=0x7f856c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.461+0000 7f855ffff640 1 -- 172.21.15.31:0/2143513698 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8568045360 con 0x7f85781074c0 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.637+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f857807bfd0 con 0x7f85781074c0 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.638+0000 7f855ffff640 1 -- 172.21.15.31:0/2143513698 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f856802ed90 con 0x7f85781074c0 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f85600391b0 msgr2=0x7f856003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 --2- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f85600391b0 0x7f856003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f856c0098c0 tx=0x7f856c006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 msgr2=0x7f857807ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 --2- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f857807ca30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8568009940 tx=0x7f8568032ce0 comp rx=0 tx=0).stop 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 shutdown_connections 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 --2- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f85600391b0 0x7f856003b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 --2- 172.21.15.31:0/2143513698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85781074c0 0x7f857807ca30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 >> 172.21.15.31:0/2143513698 conn(0x7f8578102c10 msgr2=0x7f8578103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 shutdown_connections 2024-09-16T00:31:25.104 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:19.640+0000 7f857d045640 1 -- 172.21.15.31:0/2143513698 wait complete. 2024-09-16T00:31:25.105 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-09-16T00:31:28.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:28 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2929714629' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T00:31:28.425 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:28 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: ignoring --setuser ceph since I am not root 2024-09-16T00:31:28.426 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:28 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: ignoring --setgroup ceph since I am not root 2024-09-16T00:31:28.426 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:28 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:28.365+0000 7fcaa87c4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:31:28.426 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:28 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:28.424+0000 7fcaa87c4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:31:28.801 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:28 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:28.578+0000 7fcaa87c4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:31:28.802 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:28 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:28.647+0000 7fcaa87c4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:31:28.802 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:28 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:28.801+0000 7fcaa87c4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:31:29.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:28 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:28.948+0000 7fcaa87c4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:31:29.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:29 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2929714629' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T00:31:29.533 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:29 smithi031 ceph-mon[30956]: mgrmap e5: x(active, since 11s) 2024-09-16T00:31:29.533 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:29 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:29.275+0000 7fcaa87c4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:31:29.533 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:29 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:29.338+0000 7fcaa87c4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:31:29.533 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:29 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:29.532+0000 7fcaa87c4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:31:30.875 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:30 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:30.455+0000 7fcaa87c4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:31:30.875 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:30 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:30.526+0000 7fcaa87c4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:31:31.545 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:31 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:31.274+0000 7fcaa87c4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:31:31.545 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:31 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:31.395+0000 7fcaa87c4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:31:31.837 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:31 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:31.544+0000 7fcaa87c4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:31:31.837 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:31 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:31.608+0000 7fcaa87c4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:31:31.837 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:31 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:31.766+0000 7fcaa87c4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:31:31.837 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:31 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:31.836+0000 7fcaa87c4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:31:32.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:31 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:31.899+0000 7fcaa87c4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:31:32.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:32 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:32.065+0000 7fcaa87c4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:31:32.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.934+0000 7ffb22949640 1 Processor -- start 2024-09-16T00:31:32.240 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.935+0000 7ffb22949640 1 -- start start 2024-09-16T00:31:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.935+0000 7ffb22949640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c106180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.935+0000 7ffb22949640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c075900 con 0x7ffb1c105d80 2024-09-16T00:31:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.935+0000 7ffb1bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c106180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.935+0000 7ffb1bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c106180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42632/0 (socket says 172.21.15.31:42632) 2024-09-16T00:31:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.935+0000 7ffb1bfff640 1 -- 172.21.15.31:0/2430497164 learned_addr learned my addr 172.21.15.31:0/2430497164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.935+0000 7ffb1bfff640 1 -- 172.21.15.31:0/2430497164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c1066c0 con 0x7ffb1c105d80 2024-09-16T00:31:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.936+0000 7ffb1bfff640 1 --2- 172.21.15.31:0/2430497164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c106180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009820 tx=0x7ffb0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f512789acb11fb53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:32.242 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.936+0000 7ffb1affd640 1 -- 172.21.15.31:0/2430497164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb0c037070 con 0x7ffb1c105d80 2024-09-16T00:31:32.242 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.936+0000 7ffb1affd640 1 -- 172.21.15.31:0/2430497164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb0c032900 con 0x7ffb1c105d80 2024-09-16T00:31:32.242 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.936+0000 7ffb1affd640 1 -- 172.21.15.31:0/2430497164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb0c030850 con 0x7ffb1c105d80 2024-09-16T00:31:32.242 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.937+0000 7ffb22949640 1 -- 172.21.15.31:0/2430497164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 msgr2=0x7ffb1c106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:32.242 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.937+0000 7ffb22949640 1 --2- 172.21.15.31:0/2430497164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c106180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009820 tx=0x7ffb0c02a270 comp rx=0 tx=0).stop 2024-09-16T00:31:32.242 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.937+0000 7ffb22949640 1 -- 172.21.15.31:0/2430497164 shutdown_connections 2024-09-16T00:31:32.242 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.937+0000 7ffb22949640 1 --2- 172.21.15.31:0/2430497164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c106180 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.937+0000 7ffb22949640 1 -- 172.21.15.31:0/2430497164 >> 172.21.15.31:0/2430497164 conn(0x7ffb1c101af0 msgr2=0x7ffb1c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.937+0000 7ffb22949640 1 -- 172.21.15.31:0/2430497164 shutdown_connections 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.937+0000 7ffb22949640 1 -- 172.21.15.31:0/2430497164 wait complete. 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.938+0000 7ffb22949640 1 Processor -- start 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.938+0000 7ffb22949640 1 -- start start 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.938+0000 7ffb22949640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.938+0000 7ffb22949640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1077c0 con 0x7ffb1c105d80 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.938+0000 7ffb1bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.938+0000 7ffb1bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42634/0 (socket says 172.21.15.31:42634) 2024-09-16T00:31:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.938+0000 7ffb1bfff640 1 -- 172.21.15.31:0/2929714629 learned_addr learned my addr 172.21.15.31:0/2929714629 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:32.244 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.938+0000 7ffb1bfff640 1 -- 172.21.15.31:0/2929714629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c19dce0 con 0x7ffb1c105d80 2024-09-16T00:31:32.244 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.939+0000 7ffb1bfff640 1 --2- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c19d7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0097f0 tx=0x7ffb0c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:32.244 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.939+0000 7ffb197fa640 1 -- 172.21.15.31:0/2929714629 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb0c037040 con 0x7ffb1c105d80 2024-09-16T00:31:32.244 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.939+0000 7ffb197fa640 1 -- 172.21.15.31:0/2929714629 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb0c040070 con 0x7ffb1c105d80 2024-09-16T00:31:32.244 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.939+0000 7ffb197fa640 1 -- 172.21.15.31:0/2929714629 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb0c03a390 con 0x7ffb1c105d80 2024-09-16T00:31:32.244 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.939+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c19df70 con 0x7ffb1c105d80 2024-09-16T00:31:32.244 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.939+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c1a0c60 con 0x7ffb1c105d80 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.940+0000 7ffb197fa640 1 -- 172.21.15.31:0/2929714629 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7ffb0c03a5e0 con 0x7ffb1c105d80 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.940+0000 7ffb197fa640 1 --2- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7ffb040391b0 0x7ffb0403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.940+0000 7ffb197fa640 1 -- 172.21.15.31:0/2929714629 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb0c06c5d0 con 0x7ffb1c105d80 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.940+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c106180 con 0x7ffb1c105d80 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.940+0000 7ffb1b7fe640 1 --2- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7ffb040391b0 0x7ffb0403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.941+0000 7ffb1b7fe640 1 --2- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7ffb040391b0 0x7ffb0403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb100098c0 tx=0x7ffb10006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:27.945+0000 7ffb197fa640 1 -- 172.21.15.31:0/2929714629 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb0c043760 con 0x7ffb1c105d80 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.170+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ffb1c1a0e30 con 0x7ffb1c105d80 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.221+0000 7ffb197fa640 1 -- 172.21.15.31:0/2929714629 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ffb0c06c330 con 0x7ffb1c105d80 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.222+0000 7ffb197fa640 1 -- 172.21.15.31:0/2929714629 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7ffb0c02fd40 con 0x7ffb1c105d80 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7ffb040391b0 msgr2=0x7ffb0403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:32.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 --2- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7ffb040391b0 0x7ffb0403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb100098c0 tx=0x7ffb10006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.246 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 msgr2=0x7ffb1c19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:32.246 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 --2- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c19d7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0097f0 tx=0x7ffb0c030d80 comp rx=0 tx=0).stop 2024-09-16T00:31:32.246 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 shutdown_connections 2024-09-16T00:31:32.246 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 --2- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7ffb040391b0 0x7ffb0403b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.246 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 --2- 172.21.15.31:0/2929714629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb1c105d80 0x7ffb1c19d7a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.246 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 >> 172.21.15.31:0/2929714629 conn(0x7ffb1c101af0 msgr2=0x7ffb1c191aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:32.246 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 shutdown_connections 2024-09-16T00:31:32.246 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:28.224+0000 7ffb22949640 1 -- 172.21.15.31:0/2929714629 wait complete. 2024-09-16T00:31:32.425 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:32 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:32.133+0000 7fcaa87c4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:31:32.425 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:32 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:32.203+0000 7fcaa87c4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:31:32.425 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:32 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:32.300+0000 7fcaa87c4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:31:32.425 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:32 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:32.363+0000 7fcaa87c4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:31:32.425 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:32 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:32.424+0000 7fcaa87c4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:31:33.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:32 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/293882589' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:31:33.489 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:31:33.489 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.637+0000 7f5d37e98640 1 Processor -- start 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.638+0000 7f5d37e98640 1 -- start start 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.638+0000 7f5d37e98640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3010a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.638+0000 7f5d37e98640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30075920 con 0x7f5d30107f70 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.638+0000 7f5d35c34640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3010a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.638+0000 7f5d35c34640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3010a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42664/0 (socket says 172.21.15.31:42664) 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.638+0000 7f5d35c34640 1 -- 172.21.15.31:0/2953259434 learned_addr learned my addr 172.21.15.31:0/2953259434 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:33.490 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.638+0000 7f5d35c34640 1 -- 172.21.15.31:0/2953259434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3010a8a0 con 0x7f5d30107f70 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.639+0000 7f5d35c34640 1 --2- 172.21.15.31:0/2953259434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3010a360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009820 tx=0x7f5d2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580ee47c933b2269 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.639+0000 7f5d34c32640 1 -- 172.21.15.31:0/2953259434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c037070 con 0x7f5d30107f70 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.639+0000 7f5d34c32640 1 -- 172.21.15.31:0/2953259434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d2c032900 con 0x7f5d30107f70 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.639+0000 7f5d34c32640 1 -- 172.21.15.31:0/2953259434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c030850 con 0x7f5d30107f70 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.640+0000 7f5d37e98640 1 -- 172.21.15.31:0/2953259434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 msgr2=0x7f5d3010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.640+0000 7f5d37e98640 1 --2- 172.21.15.31:0/2953259434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3010a360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009820 tx=0x7f5d2c02a270 comp rx=0 tx=0).stop 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.640+0000 7f5d37e98640 1 -- 172.21.15.31:0/2953259434 shutdown_connections 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.640+0000 7f5d37e98640 1 --2- 172.21.15.31:0/2953259434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3010a360 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.640+0000 7f5d37e98640 1 -- 172.21.15.31:0/2953259434 >> 172.21.15.31:0/2953259434 conn(0x7f5d30101af0 msgr2=0x7f5d30103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.640+0000 7f5d37e98640 1 -- 172.21.15.31:0/2953259434 shutdown_connections 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.640+0000 7f5d37e98640 1 -- 172.21.15.31:0/2953259434 wait complete. 2024-09-16T00:31:33.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.641+0000 7f5d37e98640 1 Processor -- start 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.641+0000 7f5d37e98640 1 -- start start 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.641+0000 7f5d37e98640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.641+0000 7f5d37e98640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3010b9a0 con 0x7f5d30107f70 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.641+0000 7f5d35c34640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.641+0000 7f5d35c34640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3019b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42670/0 (socket says 172.21.15.31:42670) 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.641+0000 7f5d35c34640 1 -- 172.21.15.31:0/293882589 learned_addr learned my addr 172.21.15.31:0/293882589 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.642+0000 7f5d35c34640 1 -- 172.21.15.31:0/293882589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3019bb10 con 0x7f5d30107f70 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.642+0000 7f5d35c34640 1 --2- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3019b5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c032ce0 tx=0x7f5d2c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.642+0000 7f5d26ffd640 1 -- 172.21.15.31:0/293882589 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c042070 con 0x7f5d30107f70 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.642+0000 7f5d26ffd640 1 -- 172.21.15.31:0/293882589 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d2c0308c0 con 0x7f5d30107f70 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.642+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3019bda0 con 0x7f5d30107f70 2024-09-16T00:31:33.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.642+0000 7f5d26ffd640 1 -- 172.21.15.31:0/293882589 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c037040 con 0x7f5d30107f70 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.642+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3019ea90 con 0x7f5d30107f70 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.643+0000 7f5d26ffd640 1 -- 172.21.15.31:0/293882589 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f5d2c045070 con 0x7f5d30107f70 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.643+0000 7f5d26ffd640 1 --2- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f5d1c0391b0 0x7f5d1c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.643+0000 7f5d26ffd640 1 -- 172.21.15.31:0/293882589 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5d2c06c2c0 con 0x7f5d30107f70 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.643+0000 7f5d35433640 1 -- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f5d1c0391b0 msgr2=0x7f5d1c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/945234876 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.643+0000 7f5d35433640 1 --2- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f5d1c0391b0 0x7f5d1c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.643+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d301083d0 con 0x7f5d30107f70 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.648+0000 7f5d26ffd640 1 -- 172.21.15.31:0/293882589 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d2c045520 con 0x7f5d30107f70 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.844+0000 7f5d35433640 1 -- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f5d1c0391b0 msgr2=0x7f5d1c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/945234876 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.844+0000 7f5d35433640 1 --2- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f5d1c0391b0 0x7f5d1c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:31:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.872+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5d3019eef0 con 0x7f5d30107f70 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.872+0000 7f5d26ffd640 1 -- 172.21.15.31:0/293882589 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f5d2c03cd40 con 0x7f5d30107f70 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f5d1c0391b0 msgr2=0x7f5d1c03b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 --2- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f5d1c0391b0 0x7f5d1c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 msgr2=0x7f5d3019b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 --2- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3019b5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c032ce0 tx=0x7f5d2c032d10 comp rx=0 tx=0).stop 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 shutdown_connections 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 --2- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:6800/945234876,v1:172.21.15.31:6801/945234876] conn(0x7f5d1c0391b0 0x7f5d1c03b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 --2- 172.21.15.31:0/293882589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d30107f70 0x7f5d3019b5d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 >> 172.21.15.31:0/293882589 conn(0x7f5d30101af0 msgr2=0x7f5d301026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 shutdown_connections 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:32.874+0000 7f5d37e98640 1 -- 172.21.15.31:0/293882589 wait complete. 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-16T00:31:33.494 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-09-16T00:31:33.606 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:33.277+0000 7fcaa87c4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:31:34.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: Active manager daemon x restarted 2024-09-16T00:31:34.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: Activating manager daemon x 2024-09-16T00:31:34.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T00:31:34.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: mgrmap e6: x(active, starting, since 0.00344788s) 2024-09-16T00:31:34.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: Manager daemon x is now available 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:31:34.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:33 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:31:34.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.886+0000 7f22bdce3640 1 Processor -- start 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.886+0000 7f22bdce3640 1 -- start start 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.887+0000 7f22bdce3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b8107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.887+0000 7f22bdce3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8107ee0 con 0x7f22b8107510 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.887+0000 7f22b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b8107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.887+0000 7f22b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b8107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42718/0 (socket says 172.21.15.31:42718) 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.887+0000 7f22b77fe640 1 -- 172.21.15.31:0/3169636153 learned_addr learned my addr 172.21.15.31:0/3169636153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.887+0000 7f22b77fe640 1 -- 172.21.15.31:0/3169636153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b81086c0 con 0x7f22b8107510 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.888+0000 7f22b77fe640 1 --2- 172.21.15.31:0/3169636153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b8107910 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009820 tx=0x7f22a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c411ff8be44ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:34.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.888+0000 7f22b67fc640 1 -- 172.21.15.31:0/3169636153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22a8037070 con 0x7f22b8107510 2024-09-16T00:31:34.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.888+0000 7f22b67fc640 1 -- 172.21.15.31:0/3169636153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f22a8032900 con 0x7f22b8107510 2024-09-16T00:31:34.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.888+0000 7f22b67fc640 1 -- 172.21.15.31:0/3169636153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22a8030850 con 0x7f22b8107510 2024-09-16T00:31:34.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.889+0000 7f22bdce3640 1 -- 172.21.15.31:0/3169636153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 msgr2=0x7f22b8107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:34.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.889+0000 7f22bdce3640 1 --2- 172.21.15.31:0/3169636153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b8107910 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009820 tx=0x7f22a802a270 comp rx=0 tx=0).stop 2024-09-16T00:31:34.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.889+0000 7f22bdce3640 1 -- 172.21.15.31:0/3169636153 shutdown_connections 2024-09-16T00:31:34.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.889+0000 7f22bdce3640 1 --2- 172.21.15.31:0/3169636153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b8107910 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.889+0000 7f22bdce3640 1 -- 172.21.15.31:0/3169636153 >> 172.21.15.31:0/3169636153 conn(0x7f22b8102ce0 msgr2=0x7f22b8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:34.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.889+0000 7f22bdce3640 1 -- 172.21.15.31:0/3169636153 shutdown_connections 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.889+0000 7f22bdce3640 1 -- 172.21.15.31:0/3169636153 wait complete. 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.889+0000 7f22bdce3640 1 Processor -- start 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.890+0000 7f22bdce3640 1 -- start start 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.890+0000 7f22bdce3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b807db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.890+0000 7f22bdce3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f22b810ba70 con 0x7f22b8107510 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.890+0000 7f22b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b807db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.890+0000 7f22b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b807db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42726/0 (socket says 172.21.15.31:42726) 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.890+0000 7f22b77fe640 1 -- 172.21.15.31:0/1653586576 learned_addr learned my addr 172.21.15.31:0/1653586576 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.890+0000 7f22b77fe640 1 -- 172.21.15.31:0/1653586576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b807e080 con 0x7f22b8107510 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.890+0000 7f22b77fe640 1 --2- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b807db40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f22a80097f0 tx=0x7f22a8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.891+0000 7f22b4ff9640 1 -- 172.21.15.31:0/1653586576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22a8037040 con 0x7f22b8107510 2024-09-16T00:31:34.856 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.891+0000 7f22b4ff9640 1 -- 172.21.15.31:0/1653586576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f22a8040070 con 0x7f22b8107510 2024-09-16T00:31:34.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.891+0000 7f22bdce3640 1 -- 172.21.15.31:0/1653586576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b807c270 con 0x7f22b8107510 2024-09-16T00:31:34.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.891+0000 7f22b4ff9640 1 -- 172.21.15.31:0/1653586576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22a803a390 con 0x7f22b8107510 2024-09-16T00:31:34.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.891+0000 7f22bdce3640 1 -- 172.21.15.31:0/1653586576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b807c750 con 0x7f22b8107510 2024-09-16T00:31:34.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.892+0000 7f22b4ff9640 1 -- 172.21.15.31:0/1653586576 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f22a803b7c0 con 0x7f22b8107510 2024-09-16T00:31:34.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:33.893+0000 7f22b4ff9640 1 -- 172.21.15.31:0/1653586576 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f22a806bcf0 con 0x7f22b8107510 2024-09-16T00:31:34.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.283+0000 7f22b4ff9640 1 -- 172.21.15.31:0/1653586576 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f22a806aa10 con 0x7f22b8107510 2024-09-16T00:31:34.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.283+0000 7f22b4ff9640 1 --2- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f22a4039cf0 0x7f22a403c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:34.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.283+0000 7f22b4ff9640 1 -- 172.21.15.31:0/1653586576 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22a800af30 con 0x7f22a4039cf0 2024-09-16T00:31:34.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.284+0000 7f22b6ffd640 1 --2- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f22a4039cf0 0x7f22a403c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.285+0000 7f22b6ffd640 1 --2- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f22a4039cf0 0x7f22a403c0e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f22a00098c0 tx=0x7f22a0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.285+0000 7f22b4ff9640 1 -- 172.21.15.31:0/1653586576 <== mgr.14118 v2:172.21.15.31:6800/4219649043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f22a800af30 con 0x7f22a4039cf0 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.292+0000 7f22ae7fc640 1 -- 172.21.15.31:0/1653586576 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f229c002570 con 0x7f22a4039cf0 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.292+0000 7f22b4ff9640 1 -- 172.21.15.31:0/1653586576 <== mgr.14118 v2:172.21.15.31:6800/4219649043 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f229c002570 con 0x7f22a4039cf0 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.292+0000 7f22ae7fc640 1 -- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f22a4039cf0 msgr2=0x7f22a403c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.292+0000 7f22ae7fc640 1 --2- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f22a4039cf0 0x7f22a403c0e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f22a00098c0 tx=0x7f22a0006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.292+0000 7f22ae7fc640 1 -- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 msgr2=0x7f22b807db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.292+0000 7f22ae7fc640 1 --2- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b807db40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f22a80097f0 tx=0x7f22a8030d80 comp rx=0 tx=0).stop 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.293+0000 7f22ae7fc640 1 -- 172.21.15.31:0/1653586576 shutdown_connections 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.293+0000 7f22ae7fc640 1 --2- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f22a4039cf0 0x7f22a403c0e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.293+0000 7f22ae7fc640 1 --2- 172.21.15.31:0/1653586576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22b8107510 0x7f22b807db40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.293+0000 7f22ae7fc640 1 -- 172.21.15.31:0/1653586576 >> 172.21.15.31:0/1653586576 conn(0x7f22b8102ce0 msgr2=0x7f22b8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:34.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.293+0000 7f22ae7fc640 1 -- 172.21.15.31:0/1653586576 shutdown_connections 2024-09-16T00:31:34.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:34.293+0000 7f22ae7fc640 1 -- 172.21.15.31:0/1653586576 wait complete. 2024-09-16T00:31:34.859 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-09-16T00:31:34.859 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-09-16T00:31:35.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:34 smithi031 ceph-mon[30956]: Found migration_current of "None". Setting to last migration. 2024-09-16T00:31:35.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:34 smithi031 ceph-mon[30956]: mgrmap e7: x(active, since 1.00637s) 2024-09-16T00:31:35.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:34 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:35.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:34 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:35.750 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:35.529+0000 7fca15329640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:00:31:35] ENGINE Error in HTTPServer.serve 2024-09-16T00:31:35.750 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: Traceback (most recent call last): 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: self._connections.run(self.expiration_interval) 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: self._run(expiration_interval) 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: s = self.context.wrap_socket( 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: return self.sslsocket_class._create( 2024-09-16T00:31:35.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:31:35.752 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: self.do_handshake() 2024-09-16T00:31:35.752 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:31:35.752 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: self._sslobj.do_handshake() 2024-09-16T00:31:35.752 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:31:35.752 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:36.025 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-mon[30956]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T00:31:36.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-mon[30956]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T00:31:36.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:36.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:36.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:35 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.244+0000 7f818e29a640 1 Processor -- start 2024-09-16T00:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.244+0000 7f818e29a640 1 -- start start 2024-09-16T00:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.245+0000 7f818e29a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f8188107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.245+0000 7f818e29a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8188105fc0 con 0x7f8188107a80 2024-09-16T00:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.245+0000 7f8187fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f8188107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.245+0000 7f8187fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f8188107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42730/0 (socket says 172.21.15.31:42730) 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.245+0000 7f8187fff640 1 -- 172.21.15.31:0/3684142347 learned_addr learned my addr 172.21.15.31:0/3684142347 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.245+0000 7f8187fff640 1 -- 172.21.15.31:0/3684142347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8188106140 con 0x7f8188107a80 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.246+0000 7f8187fff640 1 --2- 172.21.15.31:0/3684142347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f8188107e80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8178009820 tx=0x7f817802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a5716212db5c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.246+0000 7f8186ffd640 1 -- 172.21.15.31:0/3684142347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8178037070 con 0x7f8188107a80 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.246+0000 7f8186ffd640 1 -- 172.21.15.31:0/3684142347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8178032900 con 0x7f8188107a80 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.246+0000 7f8186ffd640 1 -- 172.21.15.31:0/3684142347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8178030850 con 0x7f8188107a80 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.246+0000 7f818e29a640 1 -- 172.21.15.31:0/3684142347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 msgr2=0x7f8188107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.246+0000 7f818e29a640 1 --2- 172.21.15.31:0/3684142347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f8188107e80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8178009820 tx=0x7f817802a270 comp rx=0 tx=0).stop 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.247+0000 7f818e29a640 1 -- 172.21.15.31:0/3684142347 shutdown_connections 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.247+0000 7f818e29a640 1 --2- 172.21.15.31:0/3684142347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f8188107e80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.247+0000 7f818e29a640 1 -- 172.21.15.31:0/3684142347 >> 172.21.15.31:0/3684142347 conn(0x7f8188101af0 msgr2=0x7f8188103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.247+0000 7f818e29a640 1 -- 172.21.15.31:0/3684142347 shutdown_connections 2024-09-16T00:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.247+0000 7f818e29a640 1 -- 172.21.15.31:0/3684142347 wait complete. 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.247+0000 7f818e29a640 1 Processor -- start 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.247+0000 7f818e29a640 1 -- start start 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.248+0000 7f818e29a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f818819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.248+0000 7f818e29a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f818810a950 con 0x7f8188107a80 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.248+0000 7f8187fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f818819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.248+0000 7f8187fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f818819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42732/0 (socket says 172.21.15.31:42732) 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.248+0000 7f8187fff640 1 -- 172.21.15.31:0/203752434 learned_addr learned my addr 172.21.15.31:0/203752434 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.248+0000 7f8187fff640 1 -- 172.21.15.31:0/203752434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818819aa30 con 0x7f8188107a80 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.248+0000 7f8187fff640 1 --2- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f818819a4f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8178009940 tx=0x7f8178032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.248+0000 7f81857fa640 1 -- 172.21.15.31:0/203752434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8178042070 con 0x7f8188107a80 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.249+0000 7f81857fa640 1 -- 172.21.15.31:0/203752434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f81780308c0 con 0x7f8188107a80 2024-09-16T00:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.249+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818819acc0 con 0x7f8188107a80 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.249+0000 7f81857fa640 1 -- 172.21.15.31:0/203752434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8178037040 con 0x7f8188107a80 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.249+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818819d9b0 con 0x7f8188107a80 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.250+0000 7f81857fa640 1 -- 172.21.15.31:0/203752434 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f8178045070 con 0x7f8188107a80 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.250+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8188107f00 con 0x7f8188107a80 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.250+0000 7f81857fa640 1 --2- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f81700390a0 0x7f817003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.250+0000 7f81857fa640 1 -- 172.21.15.31:0/203752434 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f817806c1b0 con 0x7f8188107a80 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.250+0000 7f81877fe640 1 --2- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f81700390a0 0x7f817003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.250+0000 7f81877fe640 1 --2- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f81700390a0 0x7f817003b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f817c0098c0 tx=0x7f817c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.259+0000 7f81857fa640 1 -- 172.21.15.31:0/203752434 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8178045520 con 0x7f8188107a80 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.448+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8188102e90 con 0x7f81700390a0 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.452+0000 7f81857fa640 1 -- 172.21.15.31:0/203752434 <== mgr.14118 v2:172.21.15.31:6800/4219649043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8188102e90 con 0x7f81700390a0 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f81700390a0 msgr2=0x7f817003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 --2- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f81700390a0 0x7f817003b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f817c0098c0 tx=0x7f817c006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 msgr2=0x7f818819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 --2- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f818819a4f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8178009940 tx=0x7f8178032ce0 comp rx=0 tx=0).stop 2024-09-16T00:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 shutdown_connections 2024-09-16T00:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 --2- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f81700390a0 0x7f817003b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 --2- 172.21.15.31:0/203752434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8188107a80 0x7f818819a4f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 >> 172.21.15.31:0/203752434 conn(0x7f8188101af0 msgr2=0x7f8188102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 shutdown_connections 2024-09-16T00:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:35.455+0000 7f818e29a640 1 -- 172.21.15.31:0/203752434 wait complete. 2024-09-16T00:31:37.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:36 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:35] ENGINE Bus STARTING 2024-09-16T00:31:37.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:36 smithi031 ceph-mon[30956]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:36 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:35] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:36 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:35] ENGINE Error in HTTPServer.serve 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:31:37.124 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:36 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:35] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:36 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:35] ENGINE Bus STARTED 2024-09-16T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:36 smithi031 ceph-mon[30956]: mgrmap e8: x(active, since 2s) 2024-09-16T00:31:37.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.392+0000 7f1781ebe640 1 Processor -- start 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.392+0000 7f1781ebe640 1 -- start start 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.392+0000 7f1781ebe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.392+0000 7f1781ebe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f177c10a260 con 0x7f177c109890 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.392+0000 7f177b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.392+0000 7f177b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42742/0 (socket says 172.21.15.31:42742) 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.393+0000 7f177b7fe640 1 -- 172.21.15.31:0/2115425935 learned_addr learned my addr 172.21.15.31:0/2115425935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.393+0000 7f177b7fe640 1 -- 172.21.15.31:0/2115425935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c10a890 con 0x7f177c109890 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.393+0000 7f177b7fe640 1 --2- 172.21.15.31:0/2115425935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c109c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f176c009820 tx=0x7f176c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212e9448ed589bca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f177a7fc640 1 -- 172.21.15.31:0/2115425935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f176c037070 con 0x7f177c109890 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f177a7fc640 1 -- 172.21.15.31:0/2115425935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f176c032900 con 0x7f177c109890 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f177a7fc640 1 -- 172.21.15.31:0/2115425935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f176c030850 con 0x7f177c109890 2024-09-16T00:31:37.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f1781ebe640 1 -- 172.21.15.31:0/2115425935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 msgr2=0x7f177c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f1781ebe640 1 --2- 172.21.15.31:0/2115425935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c109c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f176c009820 tx=0x7f176c02a270 comp rx=0 tx=0).stop 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f1781ebe640 1 -- 172.21.15.31:0/2115425935 shutdown_connections 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f1781ebe640 1 --2- 172.21.15.31:0/2115425935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c109c90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f1781ebe640 1 -- 172.21.15.31:0/2115425935 >> 172.21.15.31:0/2115425935 conn(0x7f177c07ccc0 msgr2=0x7f177c07d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f1781ebe640 1 -- 172.21.15.31:0/2115425935 shutdown_connections 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.394+0000 7f1781ebe640 1 -- 172.21.15.31:0/2115425935 wait complete. 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.395+0000 7f1781ebe640 1 Processor -- start 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.395+0000 7f1781ebe640 1 -- start start 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.395+0000 7f1781ebe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.395+0000 7f1781ebe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f177c10dc40 con 0x7f177c109890 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.395+0000 7f177b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.395+0000 7f177b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42748/0 (socket says 172.21.15.31:42748) 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.395+0000 7f177b7fe640 1 -- 172.21.15.31:0/4045313460 learned_addr learned my addr 172.21.15.31:0/4045313460 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.396+0000 7f177b7fe640 1 -- 172.21.15.31:0/4045313460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c19dc20 con 0x7f177c109890 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.396+0000 7f177b7fe640 1 --2- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c19d6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f176c0097f0 tx=0x7f176c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.396+0000 7f1778ff9640 1 -- 172.21.15.31:0/4045313460 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f176c037040 con 0x7f177c109890 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.396+0000 7f1778ff9640 1 -- 172.21.15.31:0/4045313460 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f176c040070 con 0x7f177c109890 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.396+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c19deb0 con 0x7f177c109890 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.396+0000 7f1778ff9640 1 -- 172.21.15.31:0/4045313460 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f176c03a390 con 0x7f177c109890 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.396+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c1a0ba0 con 0x7f177c109890 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.397+0000 7f1778ff9640 1 -- 172.21.15.31:0/4045313460 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f176c03a630 con 0x7f177c109890 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.397+0000 7f1778ff9640 1 --2- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f17640391b0 0x7f176403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.397+0000 7f1778ff9640 1 -- 172.21.15.31:0/4045313460 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f176c06ca70 con 0x7f177c109890 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.397+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c109c90 con 0x7f177c109890 2024-09-16T00:31:37.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.397+0000 7f177affd640 1 --2- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f17640391b0 0x7f176403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.398+0000 7f177affd640 1 --2- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f17640391b0 0x7f176403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17700098c0 tx=0x7f1770006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.402+0000 7f1778ff9640 1 -- 172.21.15.31:0/4045313460 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f176c04a200 con 0x7f177c109890 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.578+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f177c107bd0 con 0x7f17640391b0 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.578+0000 7f1778ff9640 1 -- 172.21.15.31:0/4045313460 <== mgr.14118 v2:172.21.15.31:6800/4219649043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f177c107bd0 con 0x7f17640391b0 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f17640391b0 msgr2=0x7f176403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 --2- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f17640391b0 0x7f176403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17700098c0 tx=0x7f1770006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 msgr2=0x7f177c19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 --2- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c19d6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f176c0097f0 tx=0x7f176c030d80 comp rx=0 tx=0).stop 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 shutdown_connections 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 --2- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f17640391b0 0x7f176403b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 --2- 172.21.15.31:0/4045313460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f177c109890 0x7f177c19d6e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 >> 172.21.15.31:0/4045313460 conn(0x7f177c07ccc0 msgr2=0x7f177c1074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 shutdown_connections 2024-09-16T00:31:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:36.581+0000 7f1781ebe640 1 -- 172.21.15.31:0/4045313460 wait complete. 2024-09-16T00:31:37.135 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-09-16T00:31:38.304 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:37 smithi031 ceph-mon[30956]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:38.604 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: Generating public/private rsa key pair. 2024-09-16T00:31:38.604 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: Your identification has been saved in /tmp/tmpx709qldl/key 2024-09-16T00:31:38.604 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: Your public key has been saved in /tmp/tmpx709qldl/key.pub 2024-09-16T00:31:38.604 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: The key fingerprint is: 2024-09-16T00:31:38.604 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: SHA256:NXnxpl05zlDeGmt/kTaRjJzTDazlvKGG+hfuhnCyCgo ceph-ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:38.604 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: The key's randomart image is: 2024-09-16T00:31:38.604 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: +---[RSA 3072]----+ 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: | .... | 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: | ..oX.=| 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: | + .@+O+| 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: | . o.+O=+| 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: | S ...=X | 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: | o o +.o.o| 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: |E . * + . o| 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: | . . . o . + .| 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: | . .. ..+. | 2024-09-16T00:31:38.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: +----[SHA256]-----+ 2024-09-16T00:31:38.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.480+0000 7f898559e640 1 Processor -- start 2024-09-16T00:31:38.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.481+0000 7f898559e640 1 -- start start 2024-09-16T00:31:38.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.481+0000 7f898559e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898007c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:38.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.481+0000 7f898559e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f898007c8e0 con 0x7f898007de60 2024-09-16T00:31:38.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.481+0000 7f897effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.481+0000 7f897effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45338/0 (socket says 172.21.15.31:45338) 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.481+0000 7f897effd640 1 -- 172.21.15.31:0/1078990846 learned_addr learned my addr 172.21.15.31:0/1078990846 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.481+0000 7f897effd640 1 -- 172.21.15.31:0/1078990846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898007ca60 con 0x7f898007de60 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.482+0000 7f897effd640 1 --2- 172.21.15.31:0/1078990846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898007c310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8970009820 tx=0x7f897002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5109aa9b5f1a3d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.482+0000 7f897e7fc640 1 -- 172.21.15.31:0/1078990846 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8970037070 con 0x7f898007de60 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.482+0000 7f897e7fc640 1 -- 172.21.15.31:0/1078990846 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8970032900 con 0x7f898007de60 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.482+0000 7f897e7fc640 1 -- 172.21.15.31:0/1078990846 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8970030850 con 0x7f898007de60 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 -- 172.21.15.31:0/1078990846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 msgr2=0x7f898007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 --2- 172.21.15.31:0/1078990846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898007c310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8970009820 tx=0x7f897002a270 comp rx=0 tx=0).stop 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 -- 172.21.15.31:0/1078990846 shutdown_connections 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 --2- 172.21.15.31:0/1078990846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898007c310 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 -- 172.21.15.31:0/1078990846 >> 172.21.15.31:0/1078990846 conn(0x7f8980102ce0 msgr2=0x7f8980105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 -- 172.21.15.31:0/1078990846 shutdown_connections 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 -- 172.21.15.31:0/1078990846 wait complete. 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 Processor -- start 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 -- start start 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.483+0000 7f898559e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f898559e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f898010dae0 con 0x7f898007de60 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f897effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f897effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45350/0 (socket says 172.21.15.31:45350) 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f897effd640 1 -- 172.21.15.31:0/3372634917 learned_addr learned my addr 172.21.15.31:0/3372634917 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f897effd640 1 -- 172.21.15.31:0/3372634917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898019dc00 con 0x7f898007de60 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f897effd640 1 --2- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898019d6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8970032ce0 tx=0x7f8970032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f897cff9640 1 -- 172.21.15.31:0/3372634917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8970042070 con 0x7f898007de60 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f897cff9640 1 -- 172.21.15.31:0/3372634917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f89700308c0 con 0x7f898007de60 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898019de90 con 0x7f898007de60 2024-09-16T00:31:38.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f897cff9640 1 -- 172.21.15.31:0/3372634917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8970037040 con 0x7f898007de60 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.484+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89801a0b80 con 0x7f898007de60 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.485+0000 7f897cff9640 1 -- 172.21.15.31:0/3372634917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8970045070 con 0x7f898007de60 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.485+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898007c390 con 0x7f898007de60 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.485+0000 7f897cff9640 1 --2- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f896c039260 0x7f896c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.486+0000 7f897cff9640 1 -- 172.21.15.31:0/3372634917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f897006c530 con 0x7f898007de60 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.486+0000 7f8977fff640 1 --2- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f896c039260 0x7f896c03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.486+0000 7f8977fff640 1 --2- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f896c039260 0x7f896c03b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f89680098c0 tx=0x7f8968006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.490+0000 7f897cff9640 1 -- 172.21.15.31:0/3372634917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8970046bd0 con 0x7f898007de60 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:37.667+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8980103f60 con 0x7f896c039260 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.309+0000 7f897cff9640 1 -- 172.21.15.31:0/3372634917 <== mgr.14118 v2:172.21.15.31:6800/4219649043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8980103f60 con 0x7f896c039260 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.311+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f896c039260 msgr2=0x7f896c03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.311+0000 7f898559e640 1 --2- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f896c039260 0x7f896c03b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f89680098c0 tx=0x7f8968006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.311+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 msgr2=0x7f898019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:38.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.311+0000 7f898559e640 1 --2- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898019d6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8970032ce0 tx=0x7f8970032d10 comp rx=0 tx=0).stop 2024-09-16T00:31:38.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.311+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 shutdown_connections 2024-09-16T00:31:38.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.311+0000 7f898559e640 1 --2- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f896c039260 0x7f896c03b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.311+0000 7f898559e640 1 --2- 172.21.15.31:0/3372634917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898007de60 0x7f898019d6c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.311+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 >> 172.21.15.31:0/3372634917 conn(0x7f8980102ce0 msgr2=0x7f8980103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:38.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.311+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 shutdown_connections 2024-09-16T00:31:38.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:38.312+0000 7f898559e640 1 -- 172.21.15.31:0/3372634917 wait complete. 2024-09-16T00:31:39.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-mon[30956]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:39.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-mon[30956]: Generating ssh key... 2024-09-16T00:31:39.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:39.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:38 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChFh7FaZZ6g3zAgN1+20VHF9QX1jVGCFwuO+2VVYdrQfyajx9ygZggQaQX/elrkk/FjRkUdhOQ/4TH6H94kSG+4/c9beCTsXYN6M48rKLT+Pb3NUihv4ndeojdMwjrVSzrVhkEpRETCjzpcuWHcqe8E+Mnfn4HaDyolfWyKyJm2eEU3mEYdp3Xfbgs8rQMqaHjaaPgftlvG0D6Iz+wjxNIaMX5TYAFHNTbBbx5bl9qhdkpzriQfSmPt28lzxYKF/s9atT4kEBbDP5/ygQha9Uzy1n3yuhn2SbKLlk49qIxPl6TPfNQ4RdMyLlvOZnRQRyMGGBVIJloVQ/mTjtqer5aujA9aMvXAVyL8pT5HhSsSih1i2OZr7oHlJ+0bTKNz/rPyldkD9AUJH68crXtJ3CT/Dwv1pGBWTNcRD1zIcfCGBXzFMDzkdzk60jc8Q2tJ11Tf+kuFIbAk++zBnQJINmpSvEwsjczXKGLmnxZCkb3aUvV1V9oG+P0Pj/XvAXujTs= ceph-ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:31:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.245+0000 7f95dc1f4640 1 Processor -- start 2024-09-16T00:31:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.245+0000 7f95dc1f4640 1 -- start start 2024-09-16T00:31:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.245+0000 7f95dc1f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d4107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.245+0000 7f95dc1f4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f95d4107ee0 con 0x7f95d4107510 2024-09-16T00:31:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.245+0000 7f95d9f90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d4107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.245+0000 7f95d9f90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d4107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45354/0 (socket says 172.21.15.31:45354) 2024-09-16T00:31:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.245+0000 7f95d9f90640 1 -- 172.21.15.31:0/4241449256 learned_addr learned my addr 172.21.15.31:0/4241449256 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.246+0000 7f95d9f90640 1 -- 172.21.15.31:0/4241449256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d41086c0 con 0x7f95d4107510 2024-09-16T00:31:39.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.246+0000 7f95d9f90640 1 --2- 172.21.15.31:0/4241449256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d4107910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009820 tx=0x7f95d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255f006f6aafd85d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:39.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.246+0000 7f95d8f8e640 1 -- 172.21.15.31:0/4241449256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95d0037070 con 0x7f95d4107510 2024-09-16T00:31:39.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.246+0000 7f95d8f8e640 1 -- 172.21.15.31:0/4241449256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95d0032900 con 0x7f95d4107510 2024-09-16T00:31:39.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.246+0000 7f95d8f8e640 1 -- 172.21.15.31:0/4241449256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95d0030850 con 0x7f95d4107510 2024-09-16T00:31:39.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.247+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4241449256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 msgr2=0x7f95d4107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:39.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.247+0000 7f95dc1f4640 1 --2- 172.21.15.31:0/4241449256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d4107910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009820 tx=0x7f95d002a270 comp rx=0 tx=0).stop 2024-09-16T00:31:39.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.247+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4241449256 shutdown_connections 2024-09-16T00:31:39.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.247+0000 7f95dc1f4640 1 --2- 172.21.15.31:0/4241449256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d4107910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.247+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4241449256 >> 172.21.15.31:0/4241449256 conn(0x7f95d4102ce0 msgr2=0x7f95d4105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.247+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4241449256 shutdown_connections 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.247+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4241449256 wait complete. 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.248+0000 7f95dc1f4640 1 Processor -- start 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.248+0000 7f95dc1f4640 1 -- start start 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.248+0000 7f95dc1f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d407db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.248+0000 7f95dc1f4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f95d410ba70 con 0x7f95d4107510 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.248+0000 7f95d9f90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d407db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.248+0000 7f95d9f90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d407db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45368/0 (socket says 172.21.15.31:45368) 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.248+0000 7f95d9f90640 1 -- 172.21.15.31:0/4042811700 learned_addr learned my addr 172.21.15.31:0/4042811700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.248+0000 7f95d9f90640 1 -- 172.21.15.31:0/4042811700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d407e080 con 0x7f95d4107510 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.248+0000 7f95d9f90640 1 --2- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d407db40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009940 tx=0x7f95d0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.249+0000 7f95caffd640 1 -- 172.21.15.31:0/4042811700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95d0042070 con 0x7f95d4107510 2024-09-16T00:31:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.249+0000 7f95caffd640 1 -- 172.21.15.31:0/4042811700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95d00308c0 con 0x7f95d4107510 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.249+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d407c270 con 0x7f95d4107510 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.249+0000 7f95caffd640 1 -- 172.21.15.31:0/4042811700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95d0037040 con 0x7f95d4107510 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.249+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d407c7b0 con 0x7f95d4107510 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.250+0000 7f95caffd640 1 -- 172.21.15.31:0/4042811700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f95d0045070 con 0x7f95d4107510 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.250+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d410bbf0 con 0x7f95d4107510 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.250+0000 7f95caffd640 1 --2- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f95c0039200 0x7f95c003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.250+0000 7f95caffd640 1 -- 172.21.15.31:0/4042811700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95d006c360 con 0x7f95d4107510 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.250+0000 7f95d978f640 1 --2- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f95c0039200 0x7f95c003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.250+0000 7f95d978f640 1 --2- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f95c0039200 0x7f95c003b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95c40098c0 tx=0x7f95c4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.255+0000 7f95caffd640 1 -- 172.21.15.31:0/4042811700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95d004b170 con 0x7f95d4107510 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.434+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f95d407d040 con 0x7f95c0039200 2024-09-16T00:31:39.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.435+0000 7f95caffd640 1 -- 172.21.15.31:0/4042811700 <== mgr.14118 v2:172.21.15.31:6800/4219649043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f95d407d040 con 0x7f95c0039200 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f95c0039200 msgr2=0x7f95c003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 --2- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f95c0039200 0x7f95c003b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95c40098c0 tx=0x7f95c4006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 msgr2=0x7f95d407db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 --2- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d407db40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009940 tx=0x7f95d0032ce0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 shutdown_connections 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 --2- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f95c0039200 0x7f95c003b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 --2- 172.21.15.31:0/4042811700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95d4107510 0x7f95d407db40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 >> 172.21.15.31:0/4042811700 conn(0x7f95d4102ce0 msgr2=0x7f95d4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 shutdown_connections 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:39.437+0000 7f95dc1f4640 1 -- 172.21.15.31:0/4042811700 wait complete. 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:31:39.981 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T00:31:39.985 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-09-16T00:31:40.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:40 smithi031 ceph-mon[30956]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:41.594 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:41 smithi031 ceph-mon[30956]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:42.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:42 smithi031 ceph-mon[30956]: Deploying cephadm binary to smithi031 2024-09-16T00:31:45.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.369+0000 7faaf6c40640 1 Processor -- start 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.369+0000 7faaf6c40640 1 -- start start 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.369+0000 7faaf6c40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf010a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.369+0000 7faaf6c40640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faaf007be80 con 0x7faaf0107f30 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.370+0000 7faaf49dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf010a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.370+0000 7faaf49dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf010a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45378/0 (socket says 172.21.15.31:45378) 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.370+0000 7faaf49dc640 1 -- 172.21.15.31:0/71500982 learned_addr learned my addr 172.21.15.31:0/71500982 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.370+0000 7faaf49dc640 1 -- 172.21.15.31:0/71500982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf010a860 con 0x7faaf0107f30 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.371+0000 7faaf49dc640 1 --2- 172.21.15.31:0/71500982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf010a320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faae4009820 tx=0x7faae402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308e84c37723201e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.371+0000 7faaef7fe640 1 -- 172.21.15.31:0/71500982 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4037070 con 0x7faaf0107f30 2024-09-16T00:31:45.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.371+0000 7faaef7fe640 1 -- 172.21.15.31:0/71500982 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faae4032900 con 0x7faaf0107f30 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.371+0000 7faaef7fe640 1 -- 172.21.15.31:0/71500982 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4030850 con 0x7faaf0107f30 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.371+0000 7faaf6c40640 1 -- 172.21.15.31:0/71500982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 msgr2=0x7faaf010a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.371+0000 7faaf6c40640 1 --2- 172.21.15.31:0/71500982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf010a320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faae4009820 tx=0x7faae402a270 comp rx=0 tx=0).stop 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.372+0000 7faaf6c40640 1 -- 172.21.15.31:0/71500982 shutdown_connections 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.372+0000 7faaf6c40640 1 --2- 172.21.15.31:0/71500982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf010a320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.372+0000 7faaf6c40640 1 -- 172.21.15.31:0/71500982 >> 172.21.15.31:0/71500982 conn(0x7faaf0101b90 msgr2=0x7faaf0103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.372+0000 7faaf6c40640 1 -- 172.21.15.31:0/71500982 shutdown_connections 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.372+0000 7faaf6c40640 1 -- 172.21.15.31:0/71500982 wait complete. 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.372+0000 7faaf6c40640 1 Processor -- start 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.372+0000 7faaf6c40640 1 -- start start 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.372+0000 7faaf6c40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.372+0000 7faaf6c40640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faaf010b900 con 0x7faaf0107f30 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.373+0000 7faaf49dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.373+0000 7faaf49dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf019b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45388/0 (socket says 172.21.15.31:45388) 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.373+0000 7faaf49dc640 1 -- 172.21.15.31:0/2067805054 learned_addr learned my addr 172.21.15.31:0/2067805054 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.373+0000 7faaf49dc640 1 -- 172.21.15.31:0/2067805054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf019b9c0 con 0x7faaf0107f30 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.373+0000 7faaf49dc640 1 --2- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf019b480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faae4032ce0 tx=0x7faae4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.373+0000 7faaedffb640 1 -- 172.21.15.31:0/2067805054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4042070 con 0x7faaf0107f30 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.373+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf019bc50 con 0x7faaf0107f30 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.373+0000 7faaedffb640 1 -- 172.21.15.31:0/2067805054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faae40308c0 con 0x7faaf0107f30 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.373+0000 7faaedffb640 1 -- 172.21.15.31:0/2067805054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4037040 con 0x7faaf0107f30 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.374+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf019e940 con 0x7faaf0107f30 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.374+0000 7faaedffb640 1 -- 172.21.15.31:0/2067805054 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7faae4045070 con 0x7faaf0107f30 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.374+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf0108390 con 0x7faaf0107f30 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.375+0000 7faaedffb640 1 --2- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7faae00391b0 0x7faae003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.375+0000 7faaedffb640 1 -- 172.21.15.31:0/2067805054 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faae406c320 con 0x7faaf0107f30 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.375+0000 7faaeffff640 1 --2- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7faae00391b0 0x7faae003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.375+0000 7faaeffff640 1 --2- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7faae00391b0 0x7faae003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faadc0098c0 tx=0x7faadc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.380+0000 7faaedffb640 1 -- 172.21.15.31:0/2067805054 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faae403cd00 con 0x7faaf0107f30 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:40.561+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}) -- 0x7faaf0102e00 con 0x7faae00391b0 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.441+0000 7faaedffb640 1 -- 172.21.15.31:0/2067805054 <== mgr.14118 v2:172.21.15.31:6800/4219649043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7faaf0102e00 con 0x7faae00391b0 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.444+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7faae00391b0 msgr2=0x7faae003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.444+0000 7faaf6c40640 1 --2- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7faae00391b0 0x7faae003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faadc0098c0 tx=0x7faadc006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.444+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 msgr2=0x7faaf019b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.444+0000 7faaf6c40640 1 --2- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf019b480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faae4032ce0 tx=0x7faae4032d10 comp rx=0 tx=0).stop 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.444+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 shutdown_connections 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.444+0000 7faaf6c40640 1 --2- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7faae00391b0 0x7faae003b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.444+0000 7faaf6c40640 1 --2- 172.21.15.31:0/2067805054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf0107f30 0x7faaf019b480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.444+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 >> 172.21.15.31:0/2067805054 conn(0x7faaf0101b90 msgr2=0x7faaf01026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.444+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 shutdown_connections 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:44.445+0000 7faaf6c40640 1 -- 172.21.15.31:0/2067805054 wait complete. 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mon service... 2024-09-16T00:31:45.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:45 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:45.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:45 smithi031 ceph-mon[30956]: Added host smithi031 2024-09-16T00:31:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:45 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:46.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T00:31:46.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.406+0000 7f5ae2c73640 1 Processor -- start 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.407+0000 7f5ae2c73640 1 -- start start 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.407+0000 7f5ae2c73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.407+0000 7f5ae2c73640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc07c7f0 con 0x7f5adc07de00 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.407+0000 7f5ae0a0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.407+0000 7f5ae0a0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45392/0 (socket says 172.21.15.31:45392) 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.407+0000 7f5ae0a0f640 1 -- 172.21.15.31:0/1624370658 learned_addr learned my addr 172.21.15.31:0/1624370658 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.407+0000 7f5ae0a0f640 1 -- 172.21.15.31:0/1624370658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc07c970 con 0x7f5adc07de00 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.408+0000 7f5ae0a0f640 1 --2- 172.21.15.31:0/1624370658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc07c2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009820 tx=0x7f5ad002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d948d87595d621d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.408+0000 7f5adb7fe640 1 -- 172.21.15.31:0/1624370658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ad0037070 con 0x7f5adc07de00 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.408+0000 7f5adb7fe640 1 -- 172.21.15.31:0/1624370658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ad0032900 con 0x7f5adc07de00 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.408+0000 7f5adb7fe640 1 -- 172.21.15.31:0/1624370658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ad0030850 con 0x7f5adc07de00 2024-09-16T00:31:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.409+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1624370658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 msgr2=0x7f5adc07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.409+0000 7f5ae2c73640 1 --2- 172.21.15.31:0/1624370658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc07c2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009820 tx=0x7f5ad002a270 comp rx=0 tx=0).stop 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.409+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1624370658 shutdown_connections 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.409+0000 7f5ae2c73640 1 --2- 172.21.15.31:0/1624370658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc07c2b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.409+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1624370658 >> 172.21.15.31:0/1624370658 conn(0x7f5adc102ce0 msgr2=0x7f5adc105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.409+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1624370658 shutdown_connections 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.409+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1624370658 wait complete. 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.410+0000 7f5ae2c73640 1 Processor -- start 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.410+0000 7f5ae2c73640 1 -- start start 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.410+0000 7f5ae2c73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.410+0000 7f5ae2c73640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc10db40 con 0x7f5adc07de00 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.410+0000 7f5ae0a0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.410+0000 7f5ae0a0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45396/0 (socket says 172.21.15.31:45396) 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.410+0000 7f5ae0a0f640 1 -- 172.21.15.31:0/1817189731 learned_addr learned my addr 172.21.15.31:0/1817189731 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.411+0000 7f5ae0a0f640 1 -- 172.21.15.31:0/1817189731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc19dbf0 con 0x7f5adc07de00 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.411+0000 7f5ae0a0f640 1 --2- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc19d6b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00097f0 tx=0x7f5ad0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.411+0000 7f5ad9ffb640 1 -- 172.21.15.31:0/1817189731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ad0037040 con 0x7f5adc07de00 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.411+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc19de80 con 0x7f5adc07de00 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.411+0000 7f5ad9ffb640 1 -- 172.21.15.31:0/1817189731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ad0040070 con 0x7f5adc07de00 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.411+0000 7f5ad9ffb640 1 -- 172.21.15.31:0/1817189731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ad003a390 con 0x7f5adc07de00 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.411+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc1a0b70 con 0x7f5adc07de00 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.412+0000 7f5ad9ffb640 1 -- 172.21.15.31:0/1817189731 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5ad003a630 con 0x7f5adc07de00 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.412+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5adc07c330 con 0x7f5adc07de00 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.413+0000 7f5ad9ffb640 1 --2- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f5acc0391b0 0x7f5acc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.413+0000 7f5ad9ffb640 1 -- 172.21.15.31:0/1817189731 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5ad006c860 con 0x7f5adc07de00 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.413+0000 7f5adbfff640 1 --2- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f5acc0391b0 0x7f5acc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.413+0000 7f5adbfff640 1 --2- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f5acc0391b0 0x7f5acc03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80098c0 tx=0x7f5ac8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.418+0000 7f5ad9ffb640 1 -- 172.21.15.31:0/1817189731 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ad004a200 con 0x7f5adc07de00 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.624+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5adc103f80 con 0x7f5acc0391b0 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.629+0000 7f5ad9ffb640 1 -- 172.21.15.31:0/1817189731 <== mgr.14118 v2:172.21.15.31:6800/4219649043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5adc103f80 con 0x7f5acc0391b0 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f5acc0391b0 msgr2=0x7f5acc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 --2- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f5acc0391b0 0x7f5acc03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80098c0 tx=0x7f5ac8006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 msgr2=0x7f5adc19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 --2- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc19d6b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00097f0 tx=0x7f5ad0030d80 comp rx=0 tx=0).stop 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 shutdown_connections 2024-09-16T00:31:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 --2- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f5acc0391b0 0x7f5acc03b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 --2- 172.21.15.31:0/1817189731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5adc07de00 0x7f5adc19d6b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 >> 172.21.15.31:0/1817189731 conn(0x7f5adc102ce0 msgr2=0x7f5adc103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 shutdown_connections 2024-09-16T00:31:46.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:45.631+0000 7f5ae2c73640 1 -- 172.21.15.31:0/1817189731 wait complete. 2024-09-16T00:31:46.430 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mgr service... 2024-09-16T00:31:46.720 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:46 smithi031 ceph-mon[30956]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:46.721 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:46 smithi031 ceph-mon[30956]: Saving service mon spec with placement count:5 2024-09-16T00:31:46.721 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:46 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:48.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:48 smithi031 ceph-mon[30956]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:48.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:48 smithi031 ceph-mon[30956]: Saving service mgr spec with placement count:2 2024-09-16T00:31:48.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:48 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:48.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:48 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:48.270 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T00:31:48.270 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.808+0000 7f8f2c701640 1 Processor -- start 2024-09-16T00:31:48.270 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.808+0000 7f8f2c701640 1 -- start start 2024-09-16T00:31:48.270 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.808+0000 7f8f2c701640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2407c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:48.270 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.808+0000 7f8f2c701640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2407c7f0 con 0x7f8f2407de00 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.808+0000 7f8f2a49d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2407c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.808+0000 7f8f2a49d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2407c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45398/0 (socket says 172.21.15.31:45398) 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.808+0000 7f8f2a49d640 1 -- 172.21.15.31:0/4180144295 learned_addr learned my addr 172.21.15.31:0/4180144295 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.808+0000 7f8f2a49d640 1 -- 172.21.15.31:0/4180144295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2407c970 con 0x7f8f2407de00 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.809+0000 7f8f2a49d640 1 --2- 172.21.15.31:0/4180144295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2407c2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009820 tx=0x7f8f2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24176422e70141ca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.809+0000 7f8f2949b640 1 -- 172.21.15.31:0/4180144295 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f20037070 con 0x7f8f2407de00 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.809+0000 7f8f2949b640 1 -- 172.21.15.31:0/4180144295 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f20032900 con 0x7f8f2407de00 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.809+0000 7f8f2949b640 1 -- 172.21.15.31:0/4180144295 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f20030850 con 0x7f8f2407de00 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.810+0000 7f8f2c701640 1 -- 172.21.15.31:0/4180144295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 msgr2=0x7f8f2407c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.810+0000 7f8f2c701640 1 --2- 172.21.15.31:0/4180144295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2407c2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009820 tx=0x7f8f2002a270 comp rx=0 tx=0).stop 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.810+0000 7f8f2c701640 1 -- 172.21.15.31:0/4180144295 shutdown_connections 2024-09-16T00:31:48.271 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.810+0000 7f8f2c701640 1 --2- 172.21.15.31:0/4180144295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2407c2b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.810+0000 7f8f2c701640 1 -- 172.21.15.31:0/4180144295 >> 172.21.15.31:0/4180144295 conn(0x7f8f24102ce0 msgr2=0x7f8f24105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.810+0000 7f8f2c701640 1 -- 172.21.15.31:0/4180144295 shutdown_connections 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.810+0000 7f8f2c701640 1 -- 172.21.15.31:0/4180144295 wait complete. 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.811+0000 7f8f2c701640 1 Processor -- start 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.811+0000 7f8f2c701640 1 -- start start 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.811+0000 7f8f2c701640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.811+0000 7f8f2c701640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2410db40 con 0x7f8f2407de00 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.811+0000 7f8f2a49d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.811+0000 7f8f2a49d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45402/0 (socket says 172.21.15.31:45402) 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.811+0000 7f8f2a49d640 1 -- 172.21.15.31:0/1718765272 learned_addr learned my addr 172.21.15.31:0/1718765272 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.811+0000 7f8f2a49d640 1 -- 172.21.15.31:0/1718765272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2419dbf0 con 0x7f8f2407de00 2024-09-16T00:31:48.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.812+0000 7f8f2a49d640 1 --2- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2419d6b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f20032ce0 tx=0x7f8f20032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.812+0000 7f8f1b7fe640 1 -- 172.21.15.31:0/1718765272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f20042070 con 0x7f8f2407de00 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.812+0000 7f8f1b7fe640 1 -- 172.21.15.31:0/1718765272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f200308c0 con 0x7f8f2407de00 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.812+0000 7f8f1b7fe640 1 -- 172.21.15.31:0/1718765272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f20037040 con 0x7f8f2407de00 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.812+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2419de80 con 0x7f8f2407de00 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.812+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241a0b70 con 0x7f8f2407de00 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.813+0000 7f8f1b7fe640 1 -- 172.21.15.31:0/1718765272 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8f20045070 con 0x7f8f2407de00 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.813+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2407c330 con 0x7f8f2407de00 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.813+0000 7f8f1b7fe640 1 --2- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f8f10039200 0x7f8f1003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.813+0000 7f8f1b7fe640 1 -- 172.21.15.31:0/1718765272 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f2006c2b0 con 0x7f8f2407de00 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.813+0000 7f8f29c9c640 1 --2- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f8f10039200 0x7f8f1003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.814+0000 7f8f29c9c640 1 --2- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f8f10039200 0x7f8f1003b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f140098c0 tx=0x7f8f14006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:46.818+0000 7f8f1b7fe640 1 -- 172.21.15.31:0/1718765272 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f2004b170 con 0x7f8f2407de00 2024-09-16T00:31:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.010+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8f24103f80 con 0x7f8f10039200 2024-09-16T00:31:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.015+0000 7f8f1b7fe640 1 -- 172.21.15.31:0/1718765272 <== mgr.14118 v2:172.21.15.31:6800/4219649043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8f24103f80 con 0x7f8f10039200 2024-09-16T00:31:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f8f10039200 msgr2=0x7f8f1003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 --2- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f8f10039200 0x7f8f1003b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f140098c0 tx=0x7f8f14006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 msgr2=0x7f8f2419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 --2- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2419d6b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f20032ce0 tx=0x7f8f20032d10 comp rx=0 tx=0).stop 2024-09-16T00:31:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 shutdown_connections 2024-09-16T00:31:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 --2- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f8f10039200 0x7f8f1003b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 --2- 172.21.15.31:0/1718765272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f2407de00 0x7f8f2419d6b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.275 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 >> 172.21.15.31:0/1718765272 conn(0x7f8f24102ce0 msgr2=0x7f8f24103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:48.275 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 shutdown_connections 2024-09-16T00:31:48.275 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:47.017+0000 7f8f2c701640 1 -- 172.21.15.31:0/1718765272 wait complete. 2024-09-16T00:31:49.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:49 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:49.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:49 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2320569007' entity='client.admin' 2024-09-16T00:31:49.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.664+0000 7f7c5378a640 1 Processor -- start 2024-09-16T00:31:49.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.664+0000 7f7c5378a640 1 -- start start 2024-09-16T00:31:49.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.664+0000 7f7c5378a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c10a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:49.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.664+0000 7f7c5378a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c07be40 con 0x7f7c4c107f70 2024-09-16T00:31:49.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.665+0000 7f7c51526640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c10a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:49.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.665+0000 7f7c51526640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c10a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57534/0 (socket says 172.21.15.31:57534) 2024-09-16T00:31:49.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.665+0000 7f7c51526640 1 -- 172.21.15.31:0/1333714794 learned_addr learned my addr 172.21.15.31:0/1333714794 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:49.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.665+0000 7f7c51526640 1 -- 172.21.15.31:0/1333714794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c10a8a0 con 0x7f7c4c107f70 2024-09-16T00:31:49.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.665+0000 7f7c51526640 1 --2- 172.21.15.31:0/1333714794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c10a360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009820 tx=0x7f7c4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a790b8d97e75ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c43fff640 1 -- 172.21.15.31:0/1333714794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c48037070 con 0x7f7c4c107f70 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c43fff640 1 -- 172.21.15.31:0/1333714794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48032920 con 0x7f7c4c107f70 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c43fff640 1 -- 172.21.15.31:0/1333714794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c48030810 con 0x7f7c4c107f70 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c5378a640 1 -- 172.21.15.31:0/1333714794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 msgr2=0x7f7c4c10a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c5378a640 1 --2- 172.21.15.31:0/1333714794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c10a360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009820 tx=0x7f7c4802a270 comp rx=0 tx=0).stop 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c5378a640 1 -- 172.21.15.31:0/1333714794 shutdown_connections 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c5378a640 1 --2- 172.21.15.31:0/1333714794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c10a360 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c5378a640 1 -- 172.21.15.31:0/1333714794 >> 172.21.15.31:0/1333714794 conn(0x7f7c4c101b90 msgr2=0x7f7c4c103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c5378a640 1 -- 172.21.15.31:0/1333714794 shutdown_connections 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.666+0000 7f7c5378a640 1 -- 172.21.15.31:0/1333714794 wait complete. 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.667+0000 7f7c5378a640 1 Processor -- start 2024-09-16T00:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.667+0000 7f7c5378a640 1 -- start start 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.667+0000 7f7c5378a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c1077f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.667+0000 7f7c5378a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c10b920 con 0x7f7c4c107f70 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.667+0000 7f7c51526640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c1077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.667+0000 7f7c51526640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c1077f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57536/0 (socket says 172.21.15.31:57536) 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.667+0000 7f7c51526640 1 -- 172.21.15.31:0/2320569007 learned_addr learned my addr 172.21.15.31:0/2320569007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.668+0000 7f7c51526640 1 -- 172.21.15.31:0/2320569007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c107d30 con 0x7f7c4c107f70 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.668+0000 7f7c51526640 1 --2- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c1077f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7c480097f0 tx=0x7f7c480327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.668+0000 7f7c427fc640 1 -- 172.21.15.31:0/2320569007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c48037040 con 0x7f7c4c107f70 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.668+0000 7f7c427fc640 1 -- 172.21.15.31:0/2320569007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48030cc0 con 0x7f7c4c107f70 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.668+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c105f20 con 0x7f7c4c107f70 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.668+0000 7f7c427fc640 1 -- 172.21.15.31:0/2320569007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c4803a6b0 con 0x7f7c4c107f70 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.668+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c106460 con 0x7f7c4c107f70 2024-09-16T00:31:49.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.669+0000 7f7c427fc640 1 -- 172.21.15.31:0/2320569007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f7c4803baf0 con 0x7f7c4c107f70 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.669+0000 7f7c427fc640 1 --2- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f7c38039200 0x7f7c3803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.669+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4c1083d0 con 0x7f7c4c107f70 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.669+0000 7f7c427fc640 1 -- 172.21.15.31:0/2320569007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c4806c7e0 con 0x7f7c4c107f70 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.669+0000 7f7c50d25640 1 --2- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f7c38039200 0x7f7c3803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.670+0000 7f7c50d25640 1 --2- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f7c38039200 0x7f7c3803b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0098c0 tx=0x7f7c3c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.675+0000 7f7c427fc640 1 -- 172.21.15.31:0/2320569007 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c48042dc0 con 0x7f7c4c107f70 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.859+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f7c4c1066e0 con 0x7f7c4c107f70 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.862+0000 7f7c427fc640 1 -- 172.21.15.31:0/2320569007 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f7c4803b3f0 con 0x7f7c4c107f70 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f7c38039200 msgr2=0x7f7c3803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 --2- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f7c38039200 0x7f7c3803b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0098c0 tx=0x7f7c3c006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 msgr2=0x7f7c4c1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:49.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 --2- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c1077f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7c480097f0 tx=0x7f7c480327b0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.917 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 shutdown_connections 2024-09-16T00:31:49.917 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 --2- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f7c38039200 0x7f7c3803b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.917 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 --2- 172.21.15.31:0/2320569007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4c107f70 0x7f7c4c1077f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.917 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 >> 172.21.15.31:0/2320569007 conn(0x7f7c4c101b90 msgr2=0x7f7c4c102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:49.917 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 shutdown_connections 2024-09-16T00:31:49.917 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:48.864+0000 7f7c5378a640 1 -- 172.21.15.31:0/2320569007 wait complete. 2024-09-16T00:31:51.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.329+0000 7fb9932d3640 1 Processor -- start 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.330+0000 7fb9932d3640 1 -- start start 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.330+0000 7fb9932d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.330+0000 7fb9932d3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c107ee0 con 0x7fb98c107510 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.330+0000 7fb99106f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.330+0000 7fb99106f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57550/0 (socket says 172.21.15.31:57550) 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.330+0000 7fb99106f640 1 -- 172.21.15.31:0/1252129748 learned_addr learned my addr 172.21.15.31:0/1252129748 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.330+0000 7fb99106f640 1 -- 172.21.15.31:0/1252129748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c1086c0 con 0x7fb98c107510 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.331+0000 7fb99106f640 1 --2- 172.21.15.31:0/1252129748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c107910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb988009820 tx=0x7fb98802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c63d3d24a7fe9f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.331+0000 7fb983fff640 1 -- 172.21.15.31:0/1252129748 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb988037070 con 0x7fb98c107510 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.331+0000 7fb983fff640 1 -- 172.21.15.31:0/1252129748 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb988032920 con 0x7fb98c107510 2024-09-16T00:31:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.331+0000 7fb983fff640 1 -- 172.21.15.31:0/1252129748 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb988030810 con 0x7fb98c107510 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.332+0000 7fb9932d3640 1 -- 172.21.15.31:0/1252129748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 msgr2=0x7fb98c107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.332+0000 7fb9932d3640 1 --2- 172.21.15.31:0/1252129748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c107910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb988009820 tx=0x7fb98802a270 comp rx=0 tx=0).stop 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.332+0000 7fb9932d3640 1 -- 172.21.15.31:0/1252129748 shutdown_connections 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.332+0000 7fb9932d3640 1 --2- 172.21.15.31:0/1252129748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c107910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.332+0000 7fb9932d3640 1 -- 172.21.15.31:0/1252129748 >> 172.21.15.31:0/1252129748 conn(0x7fb98c102ce0 msgr2=0x7fb98c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.332+0000 7fb9932d3640 1 -- 172.21.15.31:0/1252129748 shutdown_connections 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.332+0000 7fb9932d3640 1 -- 172.21.15.31:0/1252129748 wait complete. 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.333+0000 7fb9932d3640 1 Processor -- start 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.333+0000 7fb9932d3640 1 -- start start 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.333+0000 7fb9932d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c19b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.333+0000 7fb9932d3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10ba70 con 0x7fb98c107510 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.333+0000 7fb99106f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c19b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.333+0000 7fb99106f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c19b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57564/0 (socket says 172.21.15.31:57564) 2024-09-16T00:31:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.333+0000 7fb99106f640 1 -- 172.21.15.31:0/2338719758 learned_addr learned my addr 172.21.15.31:0/2338719758 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.333+0000 7fb99106f640 1 -- 172.21.15.31:0/2338719758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c19ba30 con 0x7fb98c107510 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.334+0000 7fb99106f640 1 --2- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c19b4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb9880097f0 tx=0x7fb9880327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.334+0000 7fb9827fc640 1 -- 172.21.15.31:0/2338719758 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb988037040 con 0x7fb98c107510 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.334+0000 7fb9827fc640 1 -- 172.21.15.31:0/2338719758 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb988030cc0 con 0x7fb98c107510 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.334+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c19bcc0 con 0x7fb98c107510 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.334+0000 7fb9827fc640 1 -- 172.21.15.31:0/2338719758 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98803a640 con 0x7fb98c107510 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.334+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c19e9b0 con 0x7fb98c107510 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.335+0000 7fb9827fc640 1 -- 172.21.15.31:0/2338719758 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb98803b3f0 con 0x7fb98c107510 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.335+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c10bbf0 con 0x7fb98c107510 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.335+0000 7fb9827fc640 1 --2- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fb978039200 0x7fb97803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.335+0000 7fb9827fc640 1 -- 172.21.15.31:0/2338719758 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb98806c790 con 0x7fb98c107510 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.335+0000 7fb99086e640 1 --2- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fb978039200 0x7fb97803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.336+0000 7fb99086e640 1 --2- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fb978039200 0x7fb97803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0098c0 tx=0x7fb97c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.340+0000 7fb9827fc640 1 -- 172.21.15.31:0/2338719758 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9880328c0 con 0x7fb98c107510 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.519+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb98c107990 con 0x7fb98c107510 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.524+0000 7fb9827fc640 1 -- 172.21.15.31:0/2338719758 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb988042dc0 con 0x7fb98c107510 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fb978039200 msgr2=0x7fb97803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 --2- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fb978039200 0x7fb97803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0098c0 tx=0x7fb97c006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 msgr2=0x7fb98c19b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 --2- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c19b4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb9880097f0 tx=0x7fb9880327b0 comp rx=0 tx=0).stop 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 shutdown_connections 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 --2- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fb978039200 0x7fb97803b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 --2- 172.21.15.31:0/2338719758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb98c107510 0x7fb98c19b4f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 >> 172.21.15.31:0/2338719758 conn(0x7fb98c102ce0 msgr2=0x7fb98c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 shutdown_connections 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:50.526+0000 7fb9932d3640 1 -- 172.21.15.31:0/2338719758 wait complete. 2024-09-16T00:31:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-09-16T00:31:51.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:51 smithi031 ceph-mon[30956]: from='mgr.14118 172.21.15.31:0/4096136101' entity='mgr.x' 2024-09-16T00:31:51.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:51 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2338719758' entity='client.admin' 2024-09-16T00:31:52.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:52 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2642109238' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T00:31:52.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:52 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: ignoring --setuser ceph since I am not root 2024-09-16T00:31:52.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:52 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: ignoring --setgroup ceph since I am not root 2024-09-16T00:31:52.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:52 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:52.593+0000 7fafe7af1040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:31:52.751 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:52 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:52.652+0000 7fafe7af1040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:31:53.035 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:52 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:52.811+0000 7fafe7af1040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:31:53.036 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:52 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:52.880+0000 7fafe7af1040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:31:53.036 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:53 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:53.034+0000 7fafe7af1040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:31:53.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.406+0000 7f08f2be3640 1 Processor -- start 2024-09-16T00:31:53.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.406+0000 7f08f2be3640 1 -- start start 2024-09-16T00:31:53.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.406+0000 7f08f2be3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.406+0000 7f08f2be3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec107ee0 con 0x7f08ec107510 2024-09-16T00:31:53.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.407+0000 7f08f097f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.407+0000 7f08f097f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57570/0 (socket says 172.21.15.31:57570) 2024-09-16T00:31:53.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.407+0000 7f08f097f640 1 -- 172.21.15.31:0/1876325821 learned_addr learned my addr 172.21.15.31:0/1876325821 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.407+0000 7f08f097f640 1 -- 172.21.15.31:0/1876325821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ec1086c0 con 0x7f08ec107510 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.408+0000 7f08f097f640 1 --2- 172.21.15.31:0/1876325821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec107910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009820 tx=0x7f08dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961bedb02eb66c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.408+0000 7f08eb7fe640 1 -- 172.21.15.31:0/1876325821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08dc037070 con 0x7f08ec107510 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.408+0000 7f08eb7fe640 1 -- 172.21.15.31:0/1876325821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08dc032920 con 0x7f08ec107510 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.408+0000 7f08eb7fe640 1 -- 172.21.15.31:0/1876325821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08dc030810 con 0x7f08ec107510 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.408+0000 7f08f2be3640 1 -- 172.21.15.31:0/1876325821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 msgr2=0x7f08ec107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.408+0000 7f08f2be3640 1 --2- 172.21.15.31:0/1876325821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec107910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009820 tx=0x7f08dc02a270 comp rx=0 tx=0).stop 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.408+0000 7f08f2be3640 1 -- 172.21.15.31:0/1876325821 shutdown_connections 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.408+0000 7f08f2be3640 1 --2- 172.21.15.31:0/1876325821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec107910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.408+0000 7f08f2be3640 1 -- 172.21.15.31:0/1876325821 >> 172.21.15.31:0/1876325821 conn(0x7f08ec102ce0 msgr2=0x7f08ec105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.409+0000 7f08f2be3640 1 -- 172.21.15.31:0/1876325821 shutdown_connections 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.409+0000 7f08f2be3640 1 -- 172.21.15.31:0/1876325821 wait complete. 2024-09-16T00:31:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.409+0000 7f08f2be3640 1 Processor -- start 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.409+0000 7f08f2be3640 1 -- start start 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.409+0000 7f08f2be3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.409+0000 7f08f2be3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec10ba70 con 0x7f08ec107510 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.409+0000 7f08f097f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.409+0000 7f08f097f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57574/0 (socket says 172.21.15.31:57574) 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.409+0000 7f08f097f640 1 -- 172.21.15.31:0/2642109238 learned_addr learned my addr 172.21.15.31:0/2642109238 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.410+0000 7f08f097f640 1 -- 172.21.15.31:0/2642109238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ec19dc60 con 0x7f08ec107510 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.410+0000 7f08f097f640 1 --2- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec19d720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0097f0 tx=0x7f08dc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.410+0000 7f08e9ffb640 1 -- 172.21.15.31:0/2642109238 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08dc037040 con 0x7f08ec107510 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.410+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ec19def0 con 0x7f08ec107510 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.410+0000 7f08e9ffb640 1 -- 172.21.15.31:0/2642109238 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08dc030cc0 con 0x7f08ec107510 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.410+0000 7f08e9ffb640 1 -- 172.21.15.31:0/2642109238 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08dc03a640 con 0x7f08ec107510 2024-09-16T00:31:53.069 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.410+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ec1a0be0 con 0x7f08ec107510 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.411+0000 7f08e9ffb640 1 -- 172.21.15.31:0/2642109238 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f08dc03b3f0 con 0x7f08ec107510 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.411+0000 7f08e9ffb640 1 --2- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f08d40391b0 0x7f08d403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.411+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ec10bbf0 con 0x7f08ec107510 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.411+0000 7f08e9ffb640 1 -- 172.21.15.31:0/2642109238 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f08dc06c7d0 con 0x7f08ec107510 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.412+0000 7f08ebfff640 1 --2- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f08d40391b0 0x7f08d403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.412+0000 7f08ebfff640 1 --2- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f08d40391b0 0x7f08d403b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08e00098c0 tx=0x7f08e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.416+0000 7f08e9ffb640 1 -- 172.21.15.31:0/2642109238 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08dc042e10 con 0x7f08ec107510 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:51.643+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f08ec1a0db0 con 0x7f08ec107510 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.438+0000 7f08e9ffb640 1 -- 172.21.15.31:0/2642109238 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f08dc03b6e0 con 0x7f08ec107510 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.440+0000 7f08e9ffb640 1 -- 172.21.15.31:0/2642109238 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f08dc049ca0 con 0x7f08ec107510 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f08d40391b0 msgr2=0x7f08d403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 --2- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f08d40391b0 0x7f08d403b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08e00098c0 tx=0x7f08e0006db0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 msgr2=0x7f08ec19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 --2- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec19d720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0097f0 tx=0x7f08dc0327b0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 shutdown_connections 2024-09-16T00:31:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 --2- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f08d40391b0 0x7f08d403b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 --2- 172.21.15.31:0/2642109238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08ec107510 0x7f08ec19d720 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 >> 172.21.15.31:0/2642109238 conn(0x7f08ec102ce0 msgr2=0x7f08ec103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 shutdown_connections 2024-09-16T00:31:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:52.441+0000 7f08f2be3640 1 -- 172.21.15.31:0/2642109238 wait complete. 2024-09-16T00:31:53.289 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:53 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:53.185+0000 7fafe7af1040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:31:53.593 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:53 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2642109238' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T00:31:53.594 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:53 smithi031 ceph-mon[30956]: mgrmap e9: x(active, since 19s) 2024-09-16T00:31:53.594 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:53 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:53.528+0000 7fafe7af1040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:31:53.594 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:53 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:53.592+0000 7fafe7af1040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:31:53.875 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:53 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:53.797+0000 7fafe7af1040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:31:54.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:31:54.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.458+0000 7fec88f94640 1 Processor -- start 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.458+0000 7fec88f94640 1 -- start start 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.459+0000 7fec88f94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec8410a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.459+0000 7fec88f94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec8410a880 con 0x7fec84107f50 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.459+0000 7fec8259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec8410a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.459+0000 7fec8259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec8410a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57608/0 (socket says 172.21.15.31:57608) 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.459+0000 7fec8259c640 1 -- 172.21.15.31:0/3017385036 learned_addr learned my addr 172.21.15.31:0/3017385036 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.459+0000 7fec8259c640 1 -- 172.21.15.31:0/3017385036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8410aa00 con 0x7fec84107f50 2024-09-16T00:31:54.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.460+0000 7fec8259c640 1 --2- 172.21.15.31:0/3017385036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec8410a340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec74009820 tx=0x7fec7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251da71ee71793af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.460+0000 7fec81d9b640 1 -- 172.21.15.31:0/3017385036 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec74037070 con 0x7fec84107f50 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.460+0000 7fec81d9b640 1 -- 172.21.15.31:0/3017385036 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec74032920 con 0x7fec84107f50 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.460+0000 7fec81d9b640 1 -- 172.21.15.31:0/3017385036 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec74030810 con 0x7fec84107f50 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.461+0000 7fec88f94640 1 -- 172.21.15.31:0/3017385036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 msgr2=0x7fec8410a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.461+0000 7fec88f94640 1 --2- 172.21.15.31:0/3017385036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec8410a340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec74009820 tx=0x7fec7402a270 comp rx=0 tx=0).stop 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.461+0000 7fec88f94640 1 -- 172.21.15.31:0/3017385036 shutdown_connections 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.461+0000 7fec88f94640 1 --2- 172.21.15.31:0/3017385036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec8410a340 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.461+0000 7fec88f94640 1 -- 172.21.15.31:0/3017385036 >> 172.21.15.31:0/3017385036 conn(0x7fec84101af0 msgr2=0x7fec84103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.461+0000 7fec88f94640 1 -- 172.21.15.31:0/3017385036 shutdown_connections 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.461+0000 7fec88f94640 1 -- 172.21.15.31:0/3017385036 wait complete. 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.462+0000 7fec88f94640 1 Processor -- start 2024-09-16T00:31:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.462+0000 7fec88f94640 1 -- start start 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.462+0000 7fec88f94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec841077d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.462+0000 7fec88f94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec8410ba20 con 0x7fec84107f50 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.462+0000 7fec8259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec841077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.462+0000 7fec8259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec841077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57622/0 (socket says 172.21.15.31:57622) 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.462+0000 7fec8259c640 1 -- 172.21.15.31:0/4250239727 learned_addr learned my addr 172.21.15.31:0/4250239727 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.462+0000 7fec8259c640 1 -- 172.21.15.31:0/4250239727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec84107d10 con 0x7fec84107f50 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.462+0000 7fec8259c640 1 --2- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec841077d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec74030380 tx=0x7fec74032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.463+0000 7fec7b7fe640 1 -- 172.21.15.31:0/4250239727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec74037040 con 0x7fec84107f50 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.463+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec84105f00 con 0x7fec84107f50 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.463+0000 7fec7b7fe640 1 -- 172.21.15.31:0/4250239727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec74040070 con 0x7fec84107f50 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.463+0000 7fec7b7fe640 1 -- 172.21.15.31:0/4250239727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec7403a7b0 con 0x7fec84107f50 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.463+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec84106440 con 0x7fec84107f50 2024-09-16T00:31:54.360 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.464+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec841083b0 con 0x7fec84107f50 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.464+0000 7fec7b7fe640 1 -- 172.21.15.31:0/4250239727 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fec7403aa30 con 0x7fec84107f50 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.464+0000 7fec7b7fe640 1 --2- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fec70039200 0x7fec7003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.464+0000 7fec7b7fe640 1 -- 172.21.15.31:0/4250239727 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fec7406cb20 con 0x7fec84107f50 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.464+0000 7fec7bfff640 1 -- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fec70039200 msgr2=0x7fec7003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4219649043 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.464+0000 7fec7bfff640 1 --2- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fec70039200 0x7fec7003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.473+0000 7fec7b7fe640 1 -- 172.21.15.31:0/4250239727 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec74045980 con 0x7fec84107f50 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.665+0000 7fec7bfff640 1 -- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fec70039200 msgr2=0x7fec7003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4219649043 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.665+0000 7fec7bfff640 1 --2- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fec70039200 0x7fec7003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.696+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fec84106ae0 con 0x7fec84107f50 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.696+0000 7fec7b7fe640 1 -- 172.21.15.31:0/4250239727 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fec74045460 con 0x7fec84107f50 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.699+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fec70039200 msgr2=0x7fec7003b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:31:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.699+0000 7fec88f94640 1 --2- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fec70039200 0x7fec7003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.699+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 msgr2=0x7fec841077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.699+0000 7fec88f94640 1 --2- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec841077d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec74030380 tx=0x7fec74032670 comp rx=0 tx=0).stop 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.699+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 shutdown_connections 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.699+0000 7fec88f94640 1 --2- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7fec70039200 0x7fec7003b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.699+0000 7fec88f94640 1 --2- 172.21.15.31:0/4250239727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec84107f50 0x7fec841077d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.699+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 >> 172.21.15.31:0/4250239727 conn(0x7fec84101af0 msgr2=0x7fec841026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.699+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 shutdown_connections 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:53.700+0000 7fec88f94640 1 -- 172.21.15.31:0/4250239727 wait complete. 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-16T00:31:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2024-09-16T00:31:54.482 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:54 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4250239727' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:31:55.015 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:54 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:54.750+0000 7fafe7af1040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:31:55.015 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:54 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:54.824+0000 7fafe7af1040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:31:55.869 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:55.591+0000 7fafe7af1040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:31:55.869 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:55.715+0000 7fafe7af1040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:31:56.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:55.867+0000 7fafe7af1040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:31:56.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:55.931+0000 7fafe7af1040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:31:56.125 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:56.089+0000 7fafe7af1040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:31:56.462 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:56.161+0000 7fafe7af1040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:31:56.462 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:56.224+0000 7fafe7af1040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:31:56.462 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:56.392+0000 7fafe7af1040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:31:56.462 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:56.460+0000 7fafe7af1040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:31:56.741 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:56.530+0000 7fafe7af1040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:31:56.741 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:56.616+0000 7fafe7af1040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:31:56.741 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:56.678+0000 7fafe7af1040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:31:56.741 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:56.739+0000 7fafe7af1040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:31:57.819 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:57.566+0000 7fafe7af1040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:31:57.820 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: Active manager daemon x restarted 2024-09-16T00:31:57.820 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: Activating manager daemon x 2024-09-16T00:31:57.820 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T00:31:57.820 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: mgrmap e10: x(active, starting, since 0.0031387s) 2024-09-16T00:31:57.820 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:31:57.820 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:31:57.820 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:31:57.820 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:31:57.820 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:31:57.821 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: Manager daemon x is now available 2024-09-16T00:31:57.821 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:57.821 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:31:57.821 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:31:59.567 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:59.368+0000 7faf52e05640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:00:31:59] ENGINE Error in HTTPServer.serve 2024-09-16T00:31:59.567 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: Traceback (most recent call last): 2024-09-16T00:31:59.568 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:31:59.568 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: self._connections.run(self.expiration_interval) 2024-09-16T00:31:59.568 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:31:59.568 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: self._run(expiration_interval) 2024-09-16T00:31:59.568 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:31:59.568 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:31:59.569 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:31:59.569 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:31:59.569 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:31:59.569 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: s = self.context.wrap_socket( 2024-09-16T00:31:59.569 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:31:59.569 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: return self.sslsocket_class._create( 2024-09-16T00:31:59.570 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:31:59.570 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: self.do_handshake() 2024-09-16T00:31:59.570 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:31:59.570 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: self._sslobj.do_handshake() 2024-09-16T00:31:59.570 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:31:59.571 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:31:59.584 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:31:59.584 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T00:31:59.585 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:31:59.585 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:31:59.585 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.807+0000 7f0c2c350640 1 Processor -- start 2024-09-16T00:31:59.585 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.807+0000 7f0c2c350640 1 -- start start 2024-09-16T00:31:59.585 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.808+0000 7f0c2c350640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2410a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.808+0000 7f0c2c350640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24075900 con 0x7f0c24107f90 2024-09-16T00:31:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.808+0000 7f0c2a0ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2410a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.808+0000 7f0c2a0ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2410a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57636/0 (socket says 172.21.15.31:57636) 2024-09-16T00:31:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.808+0000 7f0c2a0ec640 1 -- 172.21.15.31:0/1511315288 learned_addr learned my addr 172.21.15.31:0/1511315288 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.808+0000 7f0c2a0ec640 1 -- 172.21.15.31:0/1511315288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2410a8c0 con 0x7f0c24107f90 2024-09-16T00:31:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.809+0000 7f0c2a0ec640 1 --2- 172.21.15.31:0/1511315288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2410a380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009820 tx=0x7f0c2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a83fd46f4e9808 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.809+0000 7f0c290ea640 1 -- 172.21.15.31:0/1511315288 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c20037070 con 0x7f0c24107f90 2024-09-16T00:31:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.809+0000 7f0c290ea640 1 -- 172.21.15.31:0/1511315288 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20032920 con 0x7f0c24107f90 2024-09-16T00:31:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.809+0000 7f0c290ea640 1 -- 172.21.15.31:0/1511315288 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c20030810 con 0x7f0c24107f90 2024-09-16T00:31:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.810+0000 7f0c2c350640 1 -- 172.21.15.31:0/1511315288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 msgr2=0x7f0c2410a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.810+0000 7f0c2c350640 1 --2- 172.21.15.31:0/1511315288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2410a380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009820 tx=0x7f0c2002a270 comp rx=0 tx=0).stop 2024-09-16T00:31:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.810+0000 7f0c2c350640 1 -- 172.21.15.31:0/1511315288 shutdown_connections 2024-09-16T00:31:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.810+0000 7f0c2c350640 1 --2- 172.21.15.31:0/1511315288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2410a380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.810+0000 7f0c2c350640 1 -- 172.21.15.31:0/1511315288 >> 172.21.15.31:0/1511315288 conn(0x7f0c24101af0 msgr2=0x7f0c24103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.810+0000 7f0c2c350640 1 -- 172.21.15.31:0/1511315288 shutdown_connections 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.810+0000 7f0c2c350640 1 -- 172.21.15.31:0/1511315288 wait complete. 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.811+0000 7f0c2c350640 1 Processor -- start 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.811+0000 7f0c2c350640 1 -- start start 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.811+0000 7f0c2c350640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.811+0000 7f0c2c350640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2410ba00 con 0x7f0c24107f90 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.811+0000 7f0c2a0ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.811+0000 7f0c2a0ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2419d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57652/0 (socket says 172.21.15.31:57652) 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.811+0000 7f0c2a0ec640 1 -- 172.21.15.31:0/3470996709 learned_addr learned my addr 172.21.15.31:0/3470996709 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.811+0000 7f0c2a0ec640 1 -- 172.21.15.31:0/3470996709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2419dd50 con 0x7f0c24107f90 2024-09-16T00:31:59.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.812+0000 7f0c2a0ec640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2419d810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c200097f0 tx=0x7f0c200327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.812+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c20037040 con 0x7f0c24107f90 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.812+0000 7f0c2c350640 1 -- 172.21.15.31:0/3470996709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2419dfe0 con 0x7f0c24107f90 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.812+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20030cc0 con 0x7f0c24107f90 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.812+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c2003a6b0 con 0x7f0c24107f90 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.812+0000 7f0c2c350640 1 -- 172.21.15.31:0/3470996709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c241a0cd0 con 0x7f0c24107f90 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.813+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f0c2003b3f0 con 0x7f0c24107f90 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.813+0000 7f0c1b7fe640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 0x7f0c1003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.813+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 --> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0c1003bdd0 con 0x7f0c10039200 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.813+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0c2006c840 con 0x7f0c24107f90 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.813+0000 7f0c298eb640 1 -- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 msgr2=0x7f0c1003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4219649043 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:54.813+0000 7f0c298eb640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 0x7f0c1003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:55.014+0000 7f0c298eb640 1 -- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 msgr2=0x7f0c1003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4219649043 2024-09-16T00:31:59.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:55.014+0000 7f0c298eb640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 0x7f0c1003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:55.415+0000 7f0c298eb640 1 -- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 msgr2=0x7f0c1003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4219649043 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:55.415+0000 7f0c298eb640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 0x7f0c1003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:56.216+0000 7f0c298eb640 1 -- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 msgr2=0x7f0c1003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4219649043 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:56.216+0000 7f0c298eb640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 0x7f0c1003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:57.569+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f0c2006b2f0 con 0x7f0c24107f90 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:57.569+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 msgr2=0x7f0c1003b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:57.569+0000 7f0c1b7fe640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 0x7f0c1003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.571+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f0c20038d40 con 0x7f0c24107f90 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.571+0000 7f0c1b7fe640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0c1003ccc0 0x7f0c1003f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.571+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0c20065960 con 0x7f0c1003ccc0 2024-09-16T00:31:59.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.572+0000 7f0c298eb640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0c1003ccc0 0x7f0c1003f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.573+0000 7f0c298eb640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0c1003ccc0 0x7f0c1003f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0c14003980 tx=0x7f0c14006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.573+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f0c20065960 con 0x7f0c1003ccc0 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.581+0000 7f0c2c350640 1 -- 172.21.15.31:0/3470996709 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0c2419e4c0 con 0x7f0c1003ccc0 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.581+0000 7f0c1b7fe640 1 -- 172.21.15.31:0/3470996709 <== mgr.14150 v2:172.21.15.31:6800/502885538 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f0c2419e4c0 con 0x7f0c1003ccc0 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 -- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0c1003ccc0 msgr2=0x7f0c1003f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0c1003ccc0 0x7f0c1003f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0c14003980 tx=0x7f0c14006e10 comp rx=0 tx=0).stop 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 -- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 msgr2=0x7f0c2419d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2419d810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c200097f0 tx=0x7f0c200327b0 comp rx=0 tx=0).stop 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 -- 172.21.15.31:0/3470996709 shutdown_connections 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0c1003ccc0 0x7f0c1003f0b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:59.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:6800/4219649043,v1:172.21.15.31:6801/4219649043] conn(0x7f0c10039200 0x7f0c1003b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:59.592 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 --2- 172.21.15.31:0/3470996709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24107f90 0x7f0c2419d810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:59.592 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 -- 172.21.15.31:0/3470996709 >> 172.21.15.31:0/3470996709 conn(0x7f0c24101af0 msgr2=0x7f0c241026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:59.592 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 -- 172.21.15.31:0/3470996709 shutdown_connections 2024-09-16T00:31:59.592 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:58.582+0000 7f0c2c350640 1 -- 172.21.15.31:0/3470996709 wait complete. 2024-09-16T00:31:59.592 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2024-09-16T00:31:59.592 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2024-09-16T00:31:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-mon[30956]: mgrmap e11: x(active, since 1.00645s) 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:59] ENGINE Bus STARTING 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:59] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:59] ENGINE Error in HTTPServer.serve 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:31:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:59] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T00:31:59.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:31:59 smithi031 ceph-mon[30956]: [16/Sep/2024:00:31:59] ENGINE Bus STARTED 2024-09-16T00:32:00.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T00:32:00.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.974+0000 7fecfcfc9640 1 Processor -- start 2024-09-16T00:32:00.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.974+0000 7fecfcfc9640 1 -- start start 2024-09-16T00:32:00.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.975+0000 7fecfcfc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf8107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.857 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.975+0000 7fecfcfc9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8105fc0 con 0x7fecf8107a80 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.975+0000 7fecf659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf8107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.975+0000 7fecf659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf8107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57236/0 (socket says 172.21.15.31:57236) 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.975+0000 7fecf659c640 1 -- 172.21.15.31:0/1178667389 learned_addr learned my addr 172.21.15.31:0/1178667389 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.975+0000 7fecf659c640 1 -- 172.21.15.31:0/1178667389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf8106140 con 0x7fecf8107a80 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.976+0000 7fecf659c640 1 --2- 172.21.15.31:0/1178667389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf8107e80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fece8009820 tx=0x7fece802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29de0fe7bc54bda9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.976+0000 7fecf559a640 1 -- 172.21.15.31:0/1178667389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece8037070 con 0x7fecf8107a80 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.976+0000 7fecf559a640 1 -- 172.21.15.31:0/1178667389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece8032920 con 0x7fecf8107a80 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.976+0000 7fecf559a640 1 -- 172.21.15.31:0/1178667389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece8030810 con 0x7fecf8107a80 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.977+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1178667389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 msgr2=0x7fecf8107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.977+0000 7fecfcfc9640 1 --2- 172.21.15.31:0/1178667389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf8107e80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fece8009820 tx=0x7fece802a270 comp rx=0 tx=0).stop 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.977+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1178667389 shutdown_connections 2024-09-16T00:32:00.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.977+0000 7fecfcfc9640 1 --2- 172.21.15.31:0/1178667389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf8107e80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.977+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1178667389 >> 172.21.15.31:0/1178667389 conn(0x7fecf8101af0 msgr2=0x7fecf8103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.977+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1178667389 shutdown_connections 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.977+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1178667389 wait complete. 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.978+0000 7fecfcfc9640 1 Processor -- start 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.978+0000 7fecfcfc9640 1 -- start start 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.978+0000 7fecfcfc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf819b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.978+0000 7fecfcfc9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fecf810a950 con 0x7fecf8107a80 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.978+0000 7fecf659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf819b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.978+0000 7fecf659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf819b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57244/0 (socket says 172.21.15.31:57244) 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.979+0000 7fecf659c640 1 -- 172.21.15.31:0/1535263810 learned_addr learned my addr 172.21.15.31:0/1535263810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:00.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.979+0000 7fecf659c640 1 -- 172.21.15.31:0/1535263810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf819bb40 con 0x7fecf8107a80 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.979+0000 7fecf659c640 1 --2- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf819b600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fece8030560 tx=0x7fece8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.979+0000 7fece77fe640 1 -- 172.21.15.31:0/1535263810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece8037040 con 0x7fecf8107a80 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.979+0000 7fece77fe640 1 -- 172.21.15.31:0/1535263810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece8040070 con 0x7fecf8107a80 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.979+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf819bdd0 con 0x7fecf8107a80 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.979+0000 7fece77fe640 1 -- 172.21.15.31:0/1535263810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece803a740 con 0x7fecf8107a80 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.979+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf819eac0 con 0x7fecf8107a80 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.980+0000 7fece77fe640 1 -- 172.21.15.31:0/1535263810 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fece803a8e0 con 0x7fecf8107a80 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.980+0000 7fece77fe640 1 --2- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fece0039140 0x7fece003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.981+0000 7fece77fe640 1 -- 172.21.15.31:0/1535263810 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fece806d870 con 0x7fecf8107a80 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.981+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf8107f00 con 0x7fecf8107a80 2024-09-16T00:32:00.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.981+0000 7fecf5d9b640 1 --2- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fece0039140 0x7fece003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.981+0000 7fecf5d9b640 1 --2- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fece0039140 0x7fece003b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fecec0098c0 tx=0x7fecec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:31:59.986+0000 7fece77fe640 1 -- 172.21.15.31:0/1535263810 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fece80313a0 con 0x7fecf8107a80 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.151+0000 7fece77fe640 1 -- 172.21.15.31:0/1535263810 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fece8045a70 con 0x7fecf8107a80 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.183+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fecf8103b80 con 0x7fece0039140 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.263+0000 7fece77fe640 1 -- 172.21.15.31:0/1535263810 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fecf8103b80 con 0x7fece0039140 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fece0039140 msgr2=0x7fece003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 --2- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fece0039140 0x7fece003b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fecec0098c0 tx=0x7fecec006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 msgr2=0x7fecf819b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 --2- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf819b600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fece8030560 tx=0x7fece8032670 comp rx=0 tx=0).stop 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 shutdown_connections 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 --2- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fece0039140 0x7fece003b600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 --2- 172.21.15.31:0/1535263810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf8107a80 0x7fecf819b600 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 >> 172.21.15.31:0/1535263810 conn(0x7fecf8101af0 msgr2=0x7fecf80752a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 shutdown_connections 2024-09-16T00:32:00.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:00.266+0000 7fecfcfc9640 1 -- 172.21.15.31:0/1535263810 wait complete. 2024-09-16T00:32:00.862 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-09-16T00:32:01.104 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:01.105 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:01 smithi031 ceph-mon[30956]: mgrmap e12: x(active, since 2s) 2024-09-16T00:32:01.105 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:01 smithi031 ceph-mon[30956]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:01.105 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:01.105 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:02.357 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:02 smithi031 ceph-mon[30956]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:02.358 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:02.616 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$k65W8VgDopFCLqmqPi.FfuGQ0AImNykeYL7WkUSg35FSg.ajJ2WDy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726446721, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T00:32:02.616 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.279+0000 7f2616ed0640 1 Processor -- start 2024-09-16T00:32:02.616 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.279+0000 7f2616ed0640 1 -- start start 2024-09-16T00:32:02.616 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.280+0000 7f2616ed0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f261010a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.616 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.280+0000 7f2616ed0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2610075900 con 0x7f2610107f90 2024-09-16T00:32:02.616 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.280+0000 7f2614c6c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f261010a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.280+0000 7f2614c6c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f261010a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57246/0 (socket says 172.21.15.31:57246) 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.280+0000 7f2614c6c640 1 -- 172.21.15.31:0/1740497176 learned_addr learned my addr 172.21.15.31:0/1740497176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.280+0000 7f2614c6c640 1 -- 172.21.15.31:0/1740497176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261010a8c0 con 0x7f2610107f90 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.281+0000 7f2614c6c640 1 --2- 172.21.15.31:0/1740497176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f261010a380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2600009820 tx=0x7f260002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f90f1cb4ba3768b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.281+0000 7f260f7fe640 1 -- 172.21.15.31:0/1740497176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2600037070 con 0x7f2610107f90 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.281+0000 7f260f7fe640 1 -- 172.21.15.31:0/1740497176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2600032920 con 0x7f2610107f90 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.281+0000 7f260f7fe640 1 -- 172.21.15.31:0/1740497176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2600030810 con 0x7f2610107f90 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.282+0000 7f2616ed0640 1 -- 172.21.15.31:0/1740497176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 msgr2=0x7f261010a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.282+0000 7f2616ed0640 1 --2- 172.21.15.31:0/1740497176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f261010a380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2600009820 tx=0x7f260002a270 comp rx=0 tx=0).stop 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.282+0000 7f2616ed0640 1 -- 172.21.15.31:0/1740497176 shutdown_connections 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.282+0000 7f2616ed0640 1 --2- 172.21.15.31:0/1740497176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f261010a380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.282+0000 7f2616ed0640 1 -- 172.21.15.31:0/1740497176 >> 172.21.15.31:0/1740497176 conn(0x7f2610101af0 msgr2=0x7f2610103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.282+0000 7f2616ed0640 1 -- 172.21.15.31:0/1740497176 shutdown_connections 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.282+0000 7f2616ed0640 1 -- 172.21.15.31:0/1740497176 wait complete. 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f2616ed0640 1 Processor -- start 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f2616ed0640 1 -- start start 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f2616ed0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f2610107810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f2616ed0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f261010ba00 con 0x7f2610107f90 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f2614c6c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f2610107810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f2614c6c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f2610107810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57250/0 (socket says 172.21.15.31:57250) 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f2614c6c640 1 -- 172.21.15.31:0/4069421881 learned_addr learned my addr 172.21.15.31:0/4069421881 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f2614c6c640 1 -- 172.21.15.31:0/4069421881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2610107d50 con 0x7f2610107f90 2024-09-16T00:32:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f2614c6c640 1 --2- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f2610107810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f26000097f0 tx=0x7f26000327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.283+0000 7f260dffb640 1 -- 172.21.15.31:0/4069421881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2600037040 con 0x7f2610107f90 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.284+0000 7f260dffb640 1 -- 172.21.15.31:0/4069421881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2600030cc0 con 0x7f2610107f90 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.284+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2610105f40 con 0x7f2610107f90 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.284+0000 7f260dffb640 1 -- 172.21.15.31:0/4069421881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f260003a6b0 con 0x7f2610107f90 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.284+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2610106480 con 0x7f2610107f90 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.285+0000 7f260dffb640 1 -- 172.21.15.31:0/4069421881 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f260003baf0 con 0x7f2610107f90 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.285+0000 7f260dffb640 1 --2- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f25f8039260 0x7f25f803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.285+0000 7f260dffb640 1 -- 172.21.15.31:0/4069421881 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f260006c8e0 con 0x7f2610107f90 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.285+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26101083f0 con 0x7f2610107f90 2024-09-16T00:32:02.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.285+0000 7f260ffff640 1 --2- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f25f8039260 0x7f25f803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.286+0000 7f260ffff640 1 --2- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f25f8039260 0x7f25f803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26040098c0 tx=0x7f2604006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.290+0000 7f260dffb640 1 -- 172.21.15.31:0/4069421881 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2600042dc0 con 0x7f2610107f90 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.487+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2610106ae0 con 0x7f25f8039260 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.736+0000 7f260dffb640 1 -- 172.21.15.31:0/4069421881 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f2610106ae0 con 0x7f25f8039260 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f25f8039260 msgr2=0x7f25f803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 --2- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f25f8039260 0x7f25f803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26040098c0 tx=0x7f2604006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 msgr2=0x7f2610107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 --2- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f2610107810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f26000097f0 tx=0x7f26000327b0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 shutdown_connections 2024-09-16T00:32:02.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 --2- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f25f8039260 0x7f25f803b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.621 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 --2- 172.21.15.31:0/4069421881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2610107f90 0x7f2610107810 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.621 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 >> 172.21.15.31:0/4069421881 conn(0x7f2610101af0 msgr2=0x7f2610102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.621 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 shutdown_connections 2024-09-16T00:32:02.621 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:01.739+0000 7f2616ed0640 1 -- 172.21.15.31:0/4069421881 wait complete. 2024-09-16T00:32:02.621 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-09-16T00:32:04.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T00:32:04.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.471+0000 7f96da383640 1 Processor -- start 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.471+0000 7f96da383640 1 -- start start 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.471+0000 7f96da383640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d4107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.471+0000 7f96da383640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96d4107ee0 con 0x7f96d4107510 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.471+0000 7f96d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d4107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.471+0000 7f96d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d4107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57252/0 (socket says 172.21.15.31:57252) 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.471+0000 7f96d3fff640 1 -- 172.21.15.31:0/1273456894 learned_addr learned my addr 172.21.15.31:0/1273456894 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.472+0000 7f96d3fff640 1 -- 172.21.15.31:0/1273456894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d41086c0 con 0x7f96d4107510 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.472+0000 7f96d3fff640 1 --2- 172.21.15.31:0/1273456894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d4107910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009820 tx=0x7f96c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81cef9ae6fc4d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96d2ffd640 1 -- 172.21.15.31:0/1273456894 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c4037070 con 0x7f96d4107510 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96d2ffd640 1 -- 172.21.15.31:0/1273456894 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c4032920 con 0x7f96d4107510 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96d2ffd640 1 -- 172.21.15.31:0/1273456894 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c4030810 con 0x7f96d4107510 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96da383640 1 -- 172.21.15.31:0/1273456894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 msgr2=0x7f96d4107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96da383640 1 --2- 172.21.15.31:0/1273456894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d4107910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009820 tx=0x7f96c402a270 comp rx=0 tx=0).stop 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96da383640 1 -- 172.21.15.31:0/1273456894 shutdown_connections 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96da383640 1 --2- 172.21.15.31:0/1273456894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d4107910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96da383640 1 -- 172.21.15.31:0/1273456894 >> 172.21.15.31:0/1273456894 conn(0x7f96d4102ce0 msgr2=0x7f96d4105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96da383640 1 -- 172.21.15.31:0/1273456894 shutdown_connections 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.473+0000 7f96da383640 1 -- 172.21.15.31:0/1273456894 wait complete. 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.474+0000 7f96da383640 1 Processor -- start 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.474+0000 7f96da383640 1 -- start start 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.474+0000 7f96da383640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d407db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.474+0000 7f96da383640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96d410ba70 con 0x7f96d4107510 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d407db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d407db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57260/0 (socket says 172.21.15.31:57260) 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96d3fff640 1 -- 172.21.15.31:0/2011657369 learned_addr learned my addr 172.21.15.31:0/2011657369 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:04.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96d3fff640 1 -- 172.21.15.31:0/2011657369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d407e080 con 0x7f96d4107510 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96d3fff640 1 --2- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d407db40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96c4030560 tx=0x7f96c4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96d17fa640 1 -- 172.21.15.31:0/2011657369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c4037040 con 0x7f96d4107510 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96d17fa640 1 -- 172.21.15.31:0/2011657369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c4040070 con 0x7f96d4107510 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d407c270 con 0x7f96d4107510 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96d17fa640 1 -- 172.21.15.31:0/2011657369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c403a740 con 0x7f96d4107510 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.475+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d407c7b0 con 0x7f96d4107510 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.476+0000 7f96d17fa640 1 -- 172.21.15.31:0/2011657369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f96c403a9b0 con 0x7f96d4107510 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.476+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d410bbf0 con 0x7f96d4107510 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.477+0000 7f96d17fa640 1 --2- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96bc039200 0x7f96bc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.477+0000 7f96d17fa640 1 -- 172.21.15.31:0/2011657369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f96c406cbb0 con 0x7f96d4107510 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.477+0000 7f96d37fe640 1 --2- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96bc039200 0x7f96bc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.477+0000 7f96d37fe640 1 --2- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96bc039200 0x7f96bc03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96c80098c0 tx=0x7f96c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:04.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.481+0000 7f96d17fa640 1 -- 172.21.15.31:0/2011657369 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96c403bce0 con 0x7f96d4107510 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.622+0000 7f96d17fa640 1 -- 172.21.15.31:0/2011657369 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f96c4045980 con 0x7f96d4107510 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.658+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f96d4107990 con 0x7f96d4107510 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.658+0000 7f96d17fa640 1 -- 172.21.15.31:0/2011657369 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f96c403adf0 con 0x7f96d4107510 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96bc039200 msgr2=0x7f96bc03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 --2- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96bc039200 0x7f96bc03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96c80098c0 tx=0x7f96c8006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 msgr2=0x7f96d407db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 --2- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d407db40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96c4030560 tx=0x7f96c4032670 comp rx=0 tx=0).stop 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 shutdown_connections 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 --2- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96bc039200 0x7f96bc03b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 --2- 172.21.15.31:0/2011657369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96d4107510 0x7f96d407db40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 >> 172.21.15.31:0/2011657369 conn(0x7f96d4102ce0 msgr2=0x7f96d4103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 shutdown_connections 2024-09-16T00:32:04.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:03.660+0000 7f96da383640 1 -- 172.21.15.31:0/2011657369 wait complete. 2024-09-16T00:32:04.182 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:32:04.182 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-16T00:32:04.189 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:32:04.189 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T00:32:04.190 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-16T00:32:04.190 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T00:32:04.192 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-09-16T00:32:04.192 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:04.192 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031.front.sepia.ceph.com:8443/ 2024-09-16T00:32:04.192 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-09-16T00:32:04.192 INFO:teuthology.orchestra.run.smithi031.stdout: Password: ryl4ql2orw 2024-09-16T00:32:04.192 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:04.192 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config directory 2024-09-16T00:32:04.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:04 smithi031 ceph-mon[30956]: mgrmap e13: x(active, since 6s) 2024-09-16T00:32:04.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:04 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2011657369' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T00:32:05.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.595+0000 7fa920c13640 1 Processor -- start 2024-09-16T00:32:05.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.596+0000 7fa920c13640 1 -- start start 2024-09-16T00:32:05.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.596+0000 7fa920c13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:05.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.596+0000 7fa920c13640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c07c7f0 con 0x7fa91c07de00 2024-09-16T00:32:05.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.596+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:05.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.596+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57274/0 (socket says 172.21.15.31:57274) 2024-09-16T00:32:05.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.596+0000 7fa91a59c640 1 -- 172.21.15.31:0/1844635288 learned_addr learned my addr 172.21.15.31:0/1844635288 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:05.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.596+0000 7fa91a59c640 1 -- 172.21.15.31:0/1844635288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c07c970 con 0x7fa91c07de00 2024-09-16T00:32:05.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.597+0000 7fa91a59c640 1 --2- 172.21.15.31:0/1844635288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c07c2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009820 tx=0x7fa90c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f3e18e52ec058 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:05.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.597+0000 7fa91959a640 1 -- 172.21.15.31:0/1844635288 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa90c037070 con 0x7fa91c07de00 2024-09-16T00:32:05.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.597+0000 7fa91959a640 1 -- 172.21.15.31:0/1844635288 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c032920 con 0x7fa91c07de00 2024-09-16T00:32:05.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.597+0000 7fa91959a640 1 -- 172.21.15.31:0/1844635288 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa90c030810 con 0x7fa91c07de00 2024-09-16T00:32:05.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.598+0000 7fa920c13640 1 -- 172.21.15.31:0/1844635288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 msgr2=0x7fa91c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:05.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.598+0000 7fa920c13640 1 --2- 172.21.15.31:0/1844635288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c07c2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009820 tx=0x7fa90c02a270 comp rx=0 tx=0).stop 2024-09-16T00:32:05.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.598+0000 7fa920c13640 1 -- 172.21.15.31:0/1844635288 shutdown_connections 2024-09-16T00:32:05.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.598+0000 7fa920c13640 1 --2- 172.21.15.31:0/1844635288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c07c2b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.598+0000 7fa920c13640 1 -- 172.21.15.31:0/1844635288 >> 172.21.15.31:0/1844635288 conn(0x7fa91c102ce0 msgr2=0x7fa91c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:05.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.598+0000 7fa920c13640 1 -- 172.21.15.31:0/1844635288 shutdown_connections 2024-09-16T00:32:05.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.598+0000 7fa920c13640 1 -- 172.21.15.31:0/1844635288 wait complete. 2024-09-16T00:32:05.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.599+0000 7fa920c13640 1 Processor -- start 2024-09-16T00:32:05.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.599+0000 7fa920c13640 1 -- start start 2024-09-16T00:32:05.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.599+0000 7fa920c13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:05.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.599+0000 7fa920c13640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c10db40 con 0x7fa91c07de00 2024-09-16T00:32:05.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.599+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:05.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.599+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57284/0 (socket says 172.21.15.31:57284) 2024-09-16T00:32:05.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.599+0000 7fa91a59c640 1 -- 172.21.15.31:0/1262582895 learned_addr learned my addr 172.21.15.31:0/1262582895 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:05.449 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.599+0000 7fa91a59c640 1 -- 172.21.15.31:0/1262582895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c19dc10 con 0x7fa91c07de00 2024-09-16T00:32:05.449 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.600+0000 7fa91a59c640 1 --2- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c19d6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0097f0 tx=0x7fa90c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.600+0000 7fa90b7fe640 1 -- 172.21.15.31:0/1262582895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa90c037040 con 0x7fa91c07de00 2024-09-16T00:32:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.600+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c19dea0 con 0x7fa91c07de00 2024-09-16T00:32:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.600+0000 7fa90b7fe640 1 -- 172.21.15.31:0/1262582895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c030cc0 con 0x7fa91c07de00 2024-09-16T00:32:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.600+0000 7fa90b7fe640 1 -- 172.21.15.31:0/1262582895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa90c03a6b0 con 0x7fa91c07de00 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.600+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1a0b90 con 0x7fa91c07de00 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.601+0000 7fa90b7fe640 1 -- 172.21.15.31:0/1262582895 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa90c03a8a0 con 0x7fa91c07de00 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.601+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91c07c330 con 0x7fa91c07de00 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.601+0000 7fa90b7fe640 1 --2- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9040392a0 0x7fa90403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.601+0000 7fa90b7fe640 1 -- 172.21.15.31:0/1262582895 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa90c06ca80 con 0x7fa91c07de00 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.601+0000 7fa919d9b640 1 --2- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9040392a0 0x7fa90403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.602+0000 7fa919d9b640 1 --2- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9040392a0 0x7fa90403b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa9100098c0 tx=0x7fa910006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.606+0000 7fa90b7fe640 1 -- 172.21.15.31:0/1262582895 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa90c03b730 con 0x7fa91c07de00 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.875+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa91c1a0d60 con 0x7fa91c07de00 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.878+0000 7fa90b7fe640 1 -- 172.21.15.31:0/1262582895 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fa90c03ace0 con 0x7fa91c07de00 2024-09-16T00:32:05.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9040392a0 msgr2=0x7fa90403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 --2- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9040392a0 0x7fa90403b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa9100098c0 tx=0x7fa910006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 msgr2=0x7fa91c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 --2- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c19d6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0097f0 tx=0x7fa90c0327b0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 shutdown_connections 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 --2- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9040392a0 0x7fa90403b760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 --2- 172.21.15.31:0/1262582895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa91c07de00 0x7fa91c19d6d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 >> 172.21.15.31:0/1262582895 conn(0x7fa91c102ce0 msgr2=0x7fa91c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 shutdown_connections 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-16T00:32:04.880+0000 7fa920c13640 1 -- 172.21.15.31:0/1262582895 wait complete. 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:05.455 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-09-16T00:32:05.479 INFO:tasks.cephadm:Fetching config... 2024-09-16T00:32:05.479 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:32:05.479 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T00:32:05.496 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T00:32:05.497 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:32:05.497 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T00:32:05.553 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T00:32:05.553 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:32:05.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T00:32:05.619 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T00:32:05.620 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:32:05.620 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T00:32:05.676 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T00:32:05.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChFh7FaZZ6g3zAgN1+20VHF9QX1jVGCFwuO+2VVYdrQfyajx9ygZggQaQX/elrkk/FjRkUdhOQ/4TH6H94kSG+4/c9beCTsXYN6M48rKLT+Pb3NUihv4ndeojdMwjrVSzrVhkEpRETCjzpcuWHcqe8E+Mnfn4HaDyolfWyKyJm2eEU3mEYdp3Xfbgs8rQMqaHjaaPgftlvG0D6Iz+wjxNIaMX5TYAFHNTbBbx5bl9qhdkpzriQfSmPt28lzxYKF/s9atT4kEBbDP5/ygQha9Uzy1n3yuhn2SbKLlk49qIxPl6TPfNQ4RdMyLlvOZnRQRyMGGBVIJloVQ/mTjtqer5aujA9aMvXAVyL8pT5HhSsSih1i2OZr7oHlJ+0bTKNz/rPyldkD9AUJH68crXtJ3CT/Dwv1pGBWTNcRD1zIcfCGBXzFMDzkdzk60jc8Q2tJ11Tf+kuFIbAk++zBnQJINmpSvEwsjczXKGLmnxZCkb3aUvV1V9oG+P0Pj/XvAXujTs= ceph-ccb601ea-73c2-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:32:05.752 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChFh7FaZZ6g3zAgN1+20VHF9QX1jVGCFwuO+2VVYdrQfyajx9ygZggQaQX/elrkk/FjRkUdhOQ/4TH6H94kSG+4/c9beCTsXYN6M48rKLT+Pb3NUihv4ndeojdMwjrVSzrVhkEpRETCjzpcuWHcqe8E+Mnfn4HaDyolfWyKyJm2eEU3mEYdp3Xfbgs8rQMqaHjaaPgftlvG0D6Iz+wjxNIaMX5TYAFHNTbBbx5bl9qhdkpzriQfSmPt28lzxYKF/s9atT4kEBbDP5/ygQha9Uzy1n3yuhn2SbKLlk49qIxPl6TPfNQ4RdMyLlvOZnRQRyMGGBVIJloVQ/mTjtqer5aujA9aMvXAVyL8pT5HhSsSih1i2OZr7oHlJ+0bTKNz/rPyldkD9AUJH68crXtJ3CT/Dwv1pGBWTNcRD1zIcfCGBXzFMDzkdzk60jc8Q2tJ11Tf+kuFIbAk++zBnQJINmpSvEwsjczXKGLmnxZCkb3aUvV1V9oG+P0Pj/XvAXujTs= ceph-ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:05.765 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChFh7FaZZ6g3zAgN1+20VHF9QX1jVGCFwuO+2VVYdrQfyajx9ygZggQaQX/elrkk/FjRkUdhOQ/4TH6H94kSG+4/c9beCTsXYN6M48rKLT+Pb3NUihv4ndeojdMwjrVSzrVhkEpRETCjzpcuWHcqe8E+Mnfn4HaDyolfWyKyJm2eEU3mEYdp3Xfbgs8rQMqaHjaaPgftlvG0D6Iz+wjxNIaMX5TYAFHNTbBbx5bl9qhdkpzriQfSmPt28lzxYKF/s9atT4kEBbDP5/ygQha9Uzy1n3yuhn2SbKLlk49qIxPl6TPfNQ4RdMyLlvOZnRQRyMGGBVIJloVQ/mTjtqer5aujA9aMvXAVyL8pT5HhSsSih1i2OZr7oHlJ+0bTKNz/rPyldkD9AUJH68crXtJ3CT/Dwv1pGBWTNcRD1zIcfCGBXzFMDzkdzk60jc8Q2tJ11Tf+kuFIbAk++zBnQJINmpSvEwsjczXKGLmnxZCkb3aUvV1V9oG+P0Pj/XvAXujTs= ceph-ccb601ea-73c2-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:32:05.808 INFO:teuthology.orchestra.run.smithi081.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChFh7FaZZ6g3zAgN1+20VHF9QX1jVGCFwuO+2VVYdrQfyajx9ygZggQaQX/elrkk/FjRkUdhOQ/4TH6H94kSG+4/c9beCTsXYN6M48rKLT+Pb3NUihv4ndeojdMwjrVSzrVhkEpRETCjzpcuWHcqe8E+Mnfn4HaDyolfWyKyJm2eEU3mEYdp3Xfbgs8rQMqaHjaaPgftlvG0D6Iz+wjxNIaMX5TYAFHNTbBbx5bl9qhdkpzriQfSmPt28lzxYKF/s9atT4kEBbDP5/ygQha9Uzy1n3yuhn2SbKLlk49qIxPl6TPfNQ4RdMyLlvOZnRQRyMGGBVIJloVQ/mTjtqer5aujA9aMvXAVyL8pT5HhSsSih1i2OZr7oHlJ+0bTKNz/rPyldkD9AUJH68crXtJ3CT/Dwv1pGBWTNcRD1zIcfCGBXzFMDzkdzk60jc8Q2tJ11Tf+kuFIbAk++zBnQJINmpSvEwsjczXKGLmnxZCkb3aUvV1V9oG+P0Pj/XvAXujTs= ceph-ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:05.820 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChFh7FaZZ6g3zAgN1+20VHF9QX1jVGCFwuO+2VVYdrQfyajx9ygZggQaQX/elrkk/FjRkUdhOQ/4TH6H94kSG+4/c9beCTsXYN6M48rKLT+Pb3NUihv4ndeojdMwjrVSzrVhkEpRETCjzpcuWHcqe8E+Mnfn4HaDyolfWyKyJm2eEU3mEYdp3Xfbgs8rQMqaHjaaPgftlvG0D6Iz+wjxNIaMX5TYAFHNTbBbx5bl9qhdkpzriQfSmPt28lzxYKF/s9atT4kEBbDP5/ygQha9Uzy1n3yuhn2SbKLlk49qIxPl6TPfNQ4RdMyLlvOZnRQRyMGGBVIJloVQ/mTjtqer5aujA9aMvXAVyL8pT5HhSsSih1i2OZr7oHlJ+0bTKNz/rPyldkD9AUJH68crXtJ3CT/Dwv1pGBWTNcRD1zIcfCGBXzFMDzkdzk60jc8Q2tJ11Tf+kuFIbAk++zBnQJINmpSvEwsjczXKGLmnxZCkb3aUvV1V9oG+P0Pj/XvAXujTs= ceph-ccb601ea-73c2-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:32:05.861 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChFh7FaZZ6g3zAgN1+20VHF9QX1jVGCFwuO+2VVYdrQfyajx9ygZggQaQX/elrkk/FjRkUdhOQ/4TH6H94kSG+4/c9beCTsXYN6M48rKLT+Pb3NUihv4ndeojdMwjrVSzrVhkEpRETCjzpcuWHcqe8E+Mnfn4HaDyolfWyKyJm2eEU3mEYdp3Xfbgs8rQMqaHjaaPgftlvG0D6Iz+wjxNIaMX5TYAFHNTbBbx5bl9qhdkpzriQfSmPt28lzxYKF/s9atT4kEBbDP5/ygQha9Uzy1n3yuhn2SbKLlk49qIxPl6TPfNQ4RdMyLlvOZnRQRyMGGBVIJloVQ/mTjtqer5aujA9aMvXAVyL8pT5HhSsSih1i2OZr7oHlJ+0bTKNz/rPyldkD9AUJH68crXtJ3CT/Dwv1pGBWTNcRD1zIcfCGBXzFMDzkdzk60jc8Q2tJ11Tf+kuFIbAk++zBnQJINmpSvEwsjczXKGLmnxZCkb3aUvV1V9oG+P0Pj/XvAXujTs= ceph-ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:05.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T00:32:06.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:05 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1262582895' entity='client.admin' 2024-09-16T00:32:06.128 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.590+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1252705102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 msgr2=0x7ff0b8100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.590+0000 7ff0c06f8640 1 --2- 172.21.15.31:0/1252705102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 0x7ff0b8100940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009880 tx=0x7ff0b402a5d0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.590+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1252705102 shutdown_connections 2024-09-16T00:32:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.590+0000 7ff0c06f8640 1 --2- 172.21.15.31:0/1252705102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 0x7ff0b8100940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.590+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1252705102 >> 172.21.15.31:0/1252705102 conn(0x7ff0b80fbcd0 msgr2=0x7ff0b80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.590+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1252705102 shutdown_connections 2024-09-16T00:32:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.590+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1252705102 wait complete. 2024-09-16T00:32:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.591+0000 7ff0c06f8640 1 Processor -- start 2024-09-16T00:32:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.591+0000 7ff0c06f8640 1 -- start start 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.591+0000 7ff0c06f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 0x7ff0b80fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.591+0000 7ff0c06f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81065f0 con 0x7ff0b8100540 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.591+0000 7ff0be494640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 0x7ff0b80fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.591+0000 7ff0be494640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 0x7ff0b80fdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57298/0 (socket says 172.21.15.31:57298) 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.591+0000 7ff0be494640 1 -- 172.21.15.31:0/1505088226 learned_addr learned my addr 172.21.15.31:0/1505088226 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.591+0000 7ff0be494640 1 -- 172.21.15.31:0/1505088226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b80fe0d0 con 0x7ff0b8100540 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.592+0000 7ff0be494640 1 --2- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 0x7ff0b80fdb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4004f60 tx=0x7ff0b40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.592+0000 7ff0af7fe640 1 -- 172.21.15.31:0/1505088226 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0b403f070 con 0x7ff0b8100540 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.592+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b80fc2c0 con 0x7ff0b8100540 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.592+0000 7ff0af7fe640 1 -- 172.21.15.31:0/1505088226 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b4004d40 con 0x7ff0b8100540 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.592+0000 7ff0af7fe640 1 -- 172.21.15.31:0/1505088226 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0b4035030 con 0x7ff0b8100540 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.592+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b80fc800 con 0x7ff0b8100540 2024-09-16T00:32:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.593+0000 7ff0af7fe640 1 -- 172.21.15.31:0/1505088226 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff0b4042070 con 0x7ff0b8100540 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.593+0000 7ff0af7fe640 1 --2- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff0a40392a0 0x7ff0a403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.593+0000 7ff0af7fe640 1 -- 172.21.15.31:0/1505088226 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff0b406d030 con 0x7ff0b8100540 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.593+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b8106d70 con 0x7ff0b8100540 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.593+0000 7ff0bdc93640 1 --2- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff0a40392a0 0x7ff0a403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.593+0000 7ff0bdc93640 1 --2- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff0a40392a0 0x7ff0a403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80098c0 tx=0x7ff0a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.598+0000 7ff0af7fe640 1 -- 172.21.15.31:0/1505088226 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0b4042360 con 0x7ff0b8100540 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.781+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7ff0b8190230 con 0x7ff0b8100540 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.783+0000 7ff0af7fe640 1 -- 172.21.15.31:0/1505088226 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7ff0b402f300 con 0x7ff0b8100540 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff0a40392a0 msgr2=0x7ff0a403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 --2- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff0a40392a0 0x7ff0a403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80098c0 tx=0x7ff0a8006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 msgr2=0x7ff0b80fdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 --2- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 0x7ff0b80fdb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4004f60 tx=0x7ff0b40048f0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 shutdown_connections 2024-09-16T00:32:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 --2- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff0a40392a0 0x7ff0a403b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 --2- 172.21.15.31:0/1505088226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8100540 0x7ff0b80fdb90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 >> 172.21.15.31:0/1505088226 conn(0x7ff0b80fbcd0 msgr2=0x7ff0b8069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 shutdown_connections 2024-09-16T00:32:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:06.786+0000 7ff0c06f8640 1 -- 172.21.15.31:0/1505088226 wait complete. 2024-09-16T00:32:07.747 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T00:32:07.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T00:32:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:08.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:07 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1505088226' entity='client.admin' 2024-09-16T00:32:08.103 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:07 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:08.103 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:07 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:08.103 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:07 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:32:08.103 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:07 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:08.103 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:07 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:08.103 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:07 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:08.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.427+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/3308191720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 msgr2=0x7fb5dc100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.427+0000 7fb5e1eb4640 1 --2- 172.21.15.31:0/3308191720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 0x7fb5dc100910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009880 tx=0x7fb5c402a5d0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.427+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/3308191720 shutdown_connections 2024-09-16T00:32:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.427+0000 7fb5e1eb4640 1 --2- 172.21.15.31:0/3308191720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 0x7fb5dc100910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.427+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/3308191720 >> 172.21.15.31:0/3308191720 conn(0x7fb5dc0fbc80 msgr2=0x7fb5dc0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.427+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/3308191720 shutdown_connections 2024-09-16T00:32:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.427+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/3308191720 wait complete. 2024-09-16T00:32:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.427+0000 7fb5e1eb4640 1 Processor -- start 2024-09-16T00:32:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.428+0000 7fb5e1eb4640 1 -- start start 2024-09-16T00:32:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.428+0000 7fb5e1eb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 0x7fb5dc0fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.428+0000 7fb5e1eb4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc106620 con 0x7fb5dc100510 2024-09-16T00:32:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.428+0000 7fb5db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 0x7fb5dc0fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.428+0000 7fb5db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 0x7fb5dc0fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38372/0 (socket says 172.21.15.31:38372) 2024-09-16T00:32:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.428+0000 7fb5db7fe640 1 -- 172.21.15.31:0/4069624614 learned_addr learned my addr 172.21.15.31:0/4069624614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.428+0000 7fb5db7fe640 1 -- 172.21.15.31:0/4069624614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc0fe0a0 con 0x7fb5dc100510 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.428+0000 7fb5db7fe640 1 --2- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 0x7fb5dc0fdb60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4004f60 tx=0x7fb5c40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.429+0000 7fb5d8ff9640 1 -- 172.21.15.31:0/4069624614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5c403f070 con 0x7fb5dc100510 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.429+0000 7fb5d8ff9640 1 -- 172.21.15.31:0/4069624614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c4004d40 con 0x7fb5dc100510 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.429+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc0fc290 con 0x7fb5dc100510 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.429+0000 7fb5d8ff9640 1 -- 172.21.15.31:0/4069624614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5c4035030 con 0x7fb5dc100510 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.429+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc0fc7d0 con 0x7fb5dc100510 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.430+0000 7fb5d8ff9640 1 -- 172.21.15.31:0/4069624614 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb5c4042070 con 0x7fb5dc100510 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.430+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5dc106da0 con 0x7fb5dc100510 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.430+0000 7fb5d8ff9640 1 --2- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb5c8039300 0x7fb5c803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.430+0000 7fb5d8ff9640 1 -- 172.21.15.31:0/4069624614 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb5c406d030 con 0x7fb5dc100510 2024-09-16T00:32:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.430+0000 7fb5daffd640 1 --2- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb5c8039300 0x7fb5c803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.430+0000 7fb5daffd640 1 --2- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb5c8039300 0x7fb5c803b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0098c0 tx=0x7fb5cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:08.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.434+0000 7fb5d8ff9640 1 -- 172.21.15.31:0/4069624614 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5c4039350 con 0x7fb5dc100510 2024-09-16T00:32:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.614+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb5dc1902d0 con 0x7fb5c8039300 2024-09-16T00:32:08.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.618+0000 7fb5d8ff9640 1 -- 172.21.15.31:0/4069624614 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb5dc1902d0 con 0x7fb5c8039300 2024-09-16T00:32:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb5c8039300 msgr2=0x7fb5c803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 --2- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb5c8039300 0x7fb5c803b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0098c0 tx=0x7fb5cc006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 msgr2=0x7fb5dc0fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 --2- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 0x7fb5dc0fdb60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4004f60 tx=0x7fb5c40048f0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 shutdown_connections 2024-09-16T00:32:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 --2- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb5c8039300 0x7fb5c803b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 --2- 172.21.15.31:0/4069624614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5dc100510 0x7fb5dc0fdb60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:08.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 >> 172.21.15.31:0/4069624614 conn(0x7fb5dc0fbc80 msgr2=0x7fb5dc06a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:08.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 shutdown_connections 2024-09-16T00:32:08.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:08.620+0000 7fb5e1eb4640 1 -- 172.21.15.31:0/4069624614 wait complete. 2024-09-16T00:32:09.348 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi081 2024-09-16T00:32:09.348 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:32:09.348 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.conf 2024-09-16T00:32:09.366 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:32:09.367 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:09.425 INFO:tasks.cephadm:Adding host smithi081 to orchestrator... 2024-09-16T00:32:09.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch host add smithi081 2024-09-16T00:32:09.692 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:09.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: Updating smithi031:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: Updating smithi031:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:10.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.121+0000 7f5917698640 1 -- 172.21.15.31:0/3315760529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 msgr2=0x7f5910107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.121+0000 7f5917698640 1 --2- 172.21.15.31:0/3315760529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 0x7f5910107a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f590c009880 tx=0x7f590c02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.121+0000 7f5917698640 1 -- 172.21.15.31:0/3315760529 shutdown_connections 2024-09-16T00:32:10.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.121+0000 7f5917698640 1 --2- 172.21.15.31:0/3315760529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 0x7f5910107a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.121+0000 7f5917698640 1 -- 172.21.15.31:0/3315760529 >> 172.21.15.31:0/3315760529 conn(0x7f59100fd290 msgr2=0x7f59100ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.121+0000 7f5917698640 1 -- 172.21.15.31:0/3315760529 shutdown_connections 2024-09-16T00:32:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.121+0000 7f5917698640 1 -- 172.21.15.31:0/3315760529 wait complete. 2024-09-16T00:32:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.122+0000 7f5917698640 1 Processor -- start 2024-09-16T00:32:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.122+0000 7f5917698640 1 -- start start 2024-09-16T00:32:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.122+0000 7f5917698640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 0x7f59100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.122+0000 7f5917698640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f591010bc90 con 0x7f5910107660 2024-09-16T00:32:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.122+0000 7f5915434640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 0x7f59100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.122+0000 7f5915434640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 0x7f59100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38388/0 (socket says 172.21.15.31:38388) 2024-09-16T00:32:10.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.122+0000 7f5915434640 1 -- 172.21.15.31:0/3274827254 learned_addr learned my addr 172.21.15.31:0/3274827254 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.122+0000 7f5915434640 1 -- 172.21.15.31:0/3274827254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59100763e0 con 0x7f5910107660 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.122+0000 7f5915434640 1 --2- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 0x7f59100744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f590c009230 tx=0x7f590c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.123+0000 7f59067fc640 1 -- 172.21.15.31:0/3274827254 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f590c03f070 con 0x7f5910107660 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.123+0000 7f59067fc640 1 -- 172.21.15.31:0/3274827254 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f590c03bc80 con 0x7f5910107660 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.123+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5910074a30 con 0x7f5910107660 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.123+0000 7f59067fc640 1 -- 172.21.15.31:0/3274827254 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f590c035030 con 0x7f5910107660 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.123+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5910074ef0 con 0x7f5910107660 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.124+0000 7f59067fc640 1 -- 172.21.15.31:0/3274827254 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f590c042070 con 0x7f5910107660 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.124+0000 7f59067fc640 1 --2- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58fc039300 0x7f58fc03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.124+0000 7f59067fc640 1 -- 172.21.15.31:0/3274827254 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f590c06cfa0 con 0x7f5910107660 2024-09-16T00:32:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.124+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5910102cd0 con 0x7f5910107660 2024-09-16T00:32:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.124+0000 7f5914c33640 1 --2- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58fc039300 0x7f58fc03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.124+0000 7f5914c33640 1 --2- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58fc039300 0x7f58fc03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59000098c0 tx=0x7f5900006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:10.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.129+0000 7f59067fc640 1 -- 172.21.15.31:0/3274827254 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f590c031dd0 con 0x7f5910107660 2024-09-16T00:32:10.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:10.306+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}) -- 0x7f5910075a20 con 0x7f58fc039300 2024-09-16T00:32:10.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:10 smithi031 ceph-mon[30956]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:12.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:12 smithi031 ceph-mon[30956]: Deploying cephadm binary to smithi081 2024-09-16T00:32:14.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.314+0000 7f59067fc640 1 -- 172.21.15.31:0/3274827254 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f5910075a20 con 0x7f58fc039300 2024-09-16T00:32:14.315 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi081' with addr '172.21.15.81' 2024-09-16T00:32:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58fc039300 msgr2=0x7f58fc03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 --2- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58fc039300 0x7f58fc03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59000098c0 tx=0x7f5900006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 msgr2=0x7f59100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 --2- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 0x7f59100744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f590c009230 tx=0x7f590c003d40 comp rx=0 tx=0).stop 2024-09-16T00:32:14.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 shutdown_connections 2024-09-16T00:32:14.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 --2- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58fc039300 0x7f58fc03b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 --2- 172.21.15.31:0/3274827254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910107660 0x7f59100744f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 >> 172.21.15.31:0/3274827254 conn(0x7f59100fd290 msgr2=0x7f59100febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 shutdown_connections 2024-09-16T00:32:14.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:14.317+0000 7f5917698640 1 -- 172.21.15.31:0/3274827254 wait complete. 2024-09-16T00:32:14.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T00:32:15.199 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:15.619 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:15 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:15.619 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:15 smithi031 ceph-mon[30956]: Added host smithi081 2024-09-16T00:32:15.619 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:15 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:15.619 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:15 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:15.619 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:15 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.628+0000 7f019a81a640 1 -- 172.21.15.31:0/3633930687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 msgr2=0x7f0194106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.628+0000 7f019a81a640 1 --2- 172.21.15.31:0/3633930687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 0x7f0194106940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0184009880 tx=0x7f018402a5d0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.628+0000 7f019a81a640 1 -- 172.21.15.31:0/3633930687 shutdown_connections 2024-09-16T00:32:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.628+0000 7f019a81a640 1 --2- 172.21.15.31:0/3633930687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 0x7f0194106940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.628+0000 7f019a81a640 1 -- 172.21.15.31:0/3633930687 >> 172.21.15.31:0/3633930687 conn(0x7f01940fc1b0 msgr2=0x7f01940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.628+0000 7f019a81a640 1 -- 172.21.15.31:0/3633930687 shutdown_connections 2024-09-16T00:32:15.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.628+0000 7f019a81a640 1 -- 172.21.15.31:0/3633930687 wait complete. 2024-09-16T00:32:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.629+0000 7f019a81a640 1 Processor -- start 2024-09-16T00:32:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.629+0000 7f019a81a640 1 -- start start 2024-09-16T00:32:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.629+0000 7f019a81a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 0x7f0194195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.629+0000 7f019a81a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f019410ab90 con 0x7f0194106560 2024-09-16T00:32:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.629+0000 7f0193fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 0x7f0194195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.629+0000 7f0193fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 0x7f0194195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38412/0 (socket says 172.21.15.31:38412) 2024-09-16T00:32:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.630+0000 7f0193fff640 1 -- 172.21.15.31:0/2782744085 learned_addr learned my addr 172.21.15.31:0/2782744085 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.630+0000 7f0193fff640 1 -- 172.21.15.31:0/2782744085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0194196260 con 0x7f0194106560 2024-09-16T00:32:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.630+0000 7f0193fff640 1 --2- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 0x7f0194195d20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0184009230 tx=0x7f0184003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.630+0000 7f01917fa640 1 -- 172.21.15.31:0/2782744085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f018403f070 con 0x7f0194106560 2024-09-16T00:32:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.630+0000 7f01917fa640 1 -- 172.21.15.31:0/2782744085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018403bd40 con 0x7f0194106560 2024-09-16T00:32:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.630+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01941964f0 con 0x7f0194106560 2024-09-16T00:32:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.630+0000 7f01917fa640 1 -- 172.21.15.31:0/2782744085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0184035030 con 0x7f0194106560 2024-09-16T00:32:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.630+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0194196950 con 0x7f0194106560 2024-09-16T00:32:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.631+0000 7f01917fa640 1 -- 172.21.15.31:0/2782744085 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f0184042070 con 0x7f0194106560 2024-09-16T00:32:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.631+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0194101bd0 con 0x7f0194106560 2024-09-16T00:32:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.631+0000 7f01917fa640 1 --2- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f017c039300 0x7f017c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.631+0000 7f01917fa640 1 -- 172.21.15.31:0/2782744085 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f018406cec0 con 0x7f0194106560 2024-09-16T00:32:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.632+0000 7f01937fe640 1 --2- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f017c039300 0x7f017c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.632+0000 7f01937fe640 1 --2- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f017c039300 0x7f017c03b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f01880098c0 tx=0x7f0188006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.636+0000 7f01917fa640 1 -- 172.21.15.31:0/2782744085 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0184036df0 con 0x7f0194106560 2024-09-16T00:32:15.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.817+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01941069c0 con 0x7f017c039300 2024-09-16T00:32:15.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.818+0000 7f01917fa640 1 -- 172.21.15.31:0/2782744085 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f01941069c0 con 0x7f017c039300 2024-09-16T00:32:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:15.818 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}] 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.820+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f017c039300 msgr2=0x7f017c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.820+0000 7f019a81a640 1 --2- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f017c039300 0x7f017c03b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f01880098c0 tx=0x7f0188006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.820+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 msgr2=0x7f0194195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.820+0000 7f019a81a640 1 --2- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 0x7f0194195d20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0184009230 tx=0x7f0184003d80 comp rx=0 tx=0).stop 2024-09-16T00:32:15.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.821+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 shutdown_connections 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.821+0000 7f019a81a640 1 --2- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f017c039300 0x7f017c03b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.821+0000 7f019a81a640 1 --2- 172.21.15.31:0/2782744085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0194106560 0x7f0194195d20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.821+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 >> 172.21.15.31:0/2782744085 conn(0x7f01940fc1b0 msgr2=0x7f01940fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.821+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 shutdown_connections 2024-09-16T00:32:15.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:15.821+0000 7f019a81a640 1 -- 172.21.15.31:0/2782744085 wait complete. 2024-09-16T00:32:16.404 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi102 2024-09-16T00:32:16.404 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:32:16.404 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.conf 2024-09-16T00:32:16.432 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:32:16.432 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:16.490 INFO:tasks.cephadm:Adding host smithi102 to orchestrator... 2024-09-16T00:32:16.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch host add smithi102 2024-09-16T00:32:16.746 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:16.867 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:16.867 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:16 smithi031 ceph-mon[30956]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:17.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.168+0000 7f8064321640 1 -- 172.21.15.31:0/3565067661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 msgr2=0x7f805c102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:17.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.168+0000 7f8064321640 1 --2- 172.21.15.31:0/3565067661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 0x7f805c102b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f804c009c20 tx=0x7f804c02ab20 comp rx=0 tx=0).stop 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.168+0000 7f8064321640 1 -- 172.21.15.31:0/3565067661 shutdown_connections 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.168+0000 7f8064321640 1 --2- 172.21.15.31:0/3565067661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 0x7f805c102b10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.168+0000 7f8064321640 1 -- 172.21.15.31:0/3565067661 >> 172.21.15.31:0/3565067661 conn(0x7f805c0fc150 msgr2=0x7f805c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.168+0000 7f8064321640 1 -- 172.21.15.31:0/3565067661 shutdown_connections 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.168+0000 7f8064321640 1 -- 172.21.15.31:0/3565067661 wait complete. 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.169+0000 7f8064321640 1 Processor -- start 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.169+0000 7f8064321640 1 -- start start 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.169+0000 7f8064321640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 0x7f805c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.169+0000 7f8064321640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f805c10bca0 con 0x7f805c102730 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.169+0000 7f80620bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 0x7f805c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.169+0000 7f80620bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 0x7f805c1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38432/0 (socket says 172.21.15.31:38432) 2024-09-16T00:32:17.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.169+0000 7f80620bd640 1 -- 172.21.15.31:0/2942444584 learned_addr learned my addr 172.21.15.31:0/2942444584 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:17.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.169+0000 7f80620bd640 1 -- 172.21.15.31:0/2942444584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c197710 con 0x7f805c102730 2024-09-16T00:32:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.170+0000 7f80620bd640 1 --2- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 0x7f805c1971d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f804c030040 tx=0x7f804c02eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.170+0000 7f805b7fe640 1 -- 172.21.15.31:0/2942444584 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c047070 con 0x7f805c102730 2024-09-16T00:32:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.170+0000 7f805b7fe640 1 -- 172.21.15.31:0/2942444584 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804c0368e0 con 0x7f805c102730 2024-09-16T00:32:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.170+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805c1979a0 con 0x7f805c102730 2024-09-16T00:32:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.170+0000 7f805b7fe640 1 -- 172.21.15.31:0/2942444584 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c042420 con 0x7f805c102730 2024-09-16T00:32:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.170+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805c197e00 con 0x7f805c102730 2024-09-16T00:32:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.171+0000 7f805b7fe640 1 -- 172.21.15.31:0/2942444584 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f804c04a070 con 0x7f805c102730 2024-09-16T00:32:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.171+0000 7f805b7fe640 1 --2- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f80440392a0 0x7f804403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.171+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805c06a7b0 con 0x7f805c102730 2024-09-16T00:32:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.171+0000 7f805b7fe640 1 -- 172.21.15.31:0/2942444584 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f804c03d030 con 0x7f805c102730 2024-09-16T00:32:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.171+0000 7f80618bc640 1 --2- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f80440392a0 0x7f804403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.171+0000 7f80618bc640 1 --2- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f80440392a0 0x7f804403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f80500098c0 tx=0x7f8050006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:17.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.176+0000 7f805b7fe640 1 -- 172.21.15.31:0/2942444584 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f804c032030 con 0x7f805c102730 2024-09-16T00:32:17.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:17.359+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}) -- 0x7f805c198560 con 0x7f80440392a0 2024-09-16T00:32:18.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:17 smithi031 ceph-mon[30956]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:19.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:19.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: Updating smithi081:/etc/ceph/ceph.conf 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:19.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:20.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:20 smithi031 ceph-mon[30956]: Deploying cephadm binary to smithi102 2024-09-16T00:32:20.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:20 smithi031 ceph-mon[30956]: Updating smithi081:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:20.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:20 smithi031 ceph-mon[30956]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:20.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:20 smithi031 ceph-mon[30956]: Updating smithi081:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:32:21.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.267+0000 7f805b7fe640 1 -- 172.21.15.31:0/2942444584 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f805c198560 con 0x7f80440392a0 2024-09-16T00:32:21.268 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi102' with addr '172.21.15.102' 2024-09-16T00:32:21.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f80440392a0 msgr2=0x7f804403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 --2- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f80440392a0 0x7f804403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f80500098c0 tx=0x7f8050006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 msgr2=0x7f805c1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 --2- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 0x7f805c1971d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f804c030040 tx=0x7f804c02eb50 comp rx=0 tx=0).stop 2024-09-16T00:32:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 shutdown_connections 2024-09-16T00:32:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 --2- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f80440392a0 0x7f804403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 --2- 172.21.15.31:0/2942444584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102730 0x7f805c1971d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 >> 172.21.15.31:0/2942444584 conn(0x7f805c0fc150 msgr2=0x7f805c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 shutdown_connections 2024-09-16T00:32:21.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:21.269+0000 7f8064321640 1 -- 172.21.15.31:0/2942444584 wait complete. 2024-09-16T00:32:21.586 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:21 smithi031 ceph-mon[30956]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:21.586 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:21 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:21.587 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:21 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:21.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T00:32:22.097 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:22.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:22 smithi031 ceph-mon[30956]: Added host smithi102 2024-09-16T00:32:22.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:22.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.546+0000 7f6663605640 1 -- 172.21.15.31:0/1959071032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 msgr2=0x7f665c106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.546+0000 7f6663605640 1 --2- 172.21.15.31:0/1959071032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 0x7f665c106940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6658009880 tx=0x7f665802a5d0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.546+0000 7f6663605640 1 -- 172.21.15.31:0/1959071032 shutdown_connections 2024-09-16T00:32:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.546+0000 7f6663605640 1 --2- 172.21.15.31:0/1959071032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 0x7f665c106940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.546+0000 7f6663605640 1 -- 172.21.15.31:0/1959071032 >> 172.21.15.31:0/1959071032 conn(0x7f665c0fc1b0 msgr2=0x7f665c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.546+0000 7f6663605640 1 -- 172.21.15.31:0/1959071032 shutdown_connections 2024-09-16T00:32:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.546+0000 7f6663605640 1 -- 172.21.15.31:0/1959071032 wait complete. 2024-09-16T00:32:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.547+0000 7f6663605640 1 Processor -- start 2024-09-16T00:32:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.547+0000 7f6663605640 1 -- start start 2024-09-16T00:32:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.547+0000 7f6663605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 0x7f665c195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.547+0000 7f6663605640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f665c10ab90 con 0x7f665c106560 2024-09-16T00:32:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.548+0000 7f66613a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 0x7f665c195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.548+0000 7f66613a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 0x7f665c195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57180/0 (socket says 172.21.15.31:57180) 2024-09-16T00:32:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.548+0000 7f66613a1640 1 -- 172.21.15.31:0/3777083876 learned_addr learned my addr 172.21.15.31:0/3777083876 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.548+0000 7f66613a1640 1 -- 172.21.15.31:0/3777083876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c196290 con 0x7f665c106560 2024-09-16T00:32:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.548+0000 7f66613a1640 1 --2- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 0x7f665c195d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6658009230 tx=0x7f6658003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.548+0000 7f66527fc640 1 -- 172.21.15.31:0/3777083876 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f665803f070 con 0x7f665c106560 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.548+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c196520 con 0x7f665c106560 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.548+0000 7f66527fc640 1 -- 172.21.15.31:0/3777083876 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665803bc80 con 0x7f665c106560 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.548+0000 7f66527fc640 1 -- 172.21.15.31:0/3777083876 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6658035030 con 0x7f665c106560 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.549+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c196980 con 0x7f665c106560 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.549+0000 7f66527fc640 1 -- 172.21.15.31:0/3777083876 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6658042070 con 0x7f665c106560 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.549+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c101bd0 con 0x7f665c106560 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.550+0000 7f66527fc640 1 --2- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f66480392a0 0x7f664803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.550+0000 7f66527fc640 1 -- 172.21.15.31:0/3777083876 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f665806cc70 con 0x7f665c106560 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.550+0000 7f6660ba0640 1 --2- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f66480392a0 0x7f664803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.550+0000 7f6660ba0640 1 --2- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f66480392a0 0x7f664803b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f664c0098c0 tx=0x7f664c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.554+0000 7f66527fc640 1 -- 172.21.15.31:0/3777083876 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f665802fdc0 con 0x7f665c106560 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.740+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f665c1069c0 con 0x7f66480392a0 2024-09-16T00:32:22.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.741+0000 7f66527fc640 1 -- 172.21.15.31:0/3777083876 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f665c1069c0 con 0x7f66480392a0 2024-09-16T00:32:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:22.741 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}] 2024-09-16T00:32:22.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f66480392a0 msgr2=0x7f664803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 --2- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f66480392a0 0x7f664803b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f664c0098c0 tx=0x7f664c006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 msgr2=0x7f665c195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 --2- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 0x7f665c195d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6658009230 tx=0x7f6658003d40 comp rx=0 tx=0).stop 2024-09-16T00:32:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 shutdown_connections 2024-09-16T00:32:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 --2- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f66480392a0 0x7f664803b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 --2- 172.21.15.31:0/3777083876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f665c106560 0x7f665c195d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 >> 172.21.15.31:0/3777083876 conn(0x7f665c0fc1b0 msgr2=0x7f665c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 shutdown_connections 2024-09-16T00:32:22.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:22.743+0000 7f6663605640 1 -- 172.21.15.31:0/3777083876 wait complete. 2024-09-16T00:32:23.312 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T00:32:23.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T00:32:23.561 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:23.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:23 smithi031 ceph-mon[30956]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:23.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.971+0000 7fb356fc2640 1 -- 172.21.15.31:0/3257067398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 msgr2=0x7fb350106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.971+0000 7fb356fc2640 1 --2- 172.21.15.31:0/3257067398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 0x7fb350106940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb344009880 tx=0x7fb34402a5d0 comp rx=0 tx=0).stop 2024-09-16T00:32:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.971+0000 7fb356fc2640 1 -- 172.21.15.31:0/3257067398 shutdown_connections 2024-09-16T00:32:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.971+0000 7fb356fc2640 1 --2- 172.21.15.31:0/3257067398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 0x7fb350106940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.971+0000 7fb356fc2640 1 -- 172.21.15.31:0/3257067398 >> 172.21.15.31:0/3257067398 conn(0x7fb3500fc1b0 msgr2=0x7fb3500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.971+0000 7fb356fc2640 1 -- 172.21.15.31:0/3257067398 shutdown_connections 2024-09-16T00:32:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.971+0000 7fb356fc2640 1 -- 172.21.15.31:0/3257067398 wait complete. 2024-09-16T00:32:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.972+0000 7fb356fc2640 1 Processor -- start 2024-09-16T00:32:23.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.972+0000 7fb356fc2640 1 -- start start 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.972+0000 7fb356fc2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 0x7fb350197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.972+0000 7fb356fc2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb35010ab90 con 0x7fb350106560 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.972+0000 7fb354d5e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 0x7fb350197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.972+0000 7fb354d5e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 0x7fb350197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57190/0 (socket says 172.21.15.31:57190) 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.972+0000 7fb354d5e640 1 -- 172.21.15.31:0/210998892 learned_addr learned my addr 172.21.15.31:0/210998892 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.972+0000 7fb354d5e640 1 -- 172.21.15.31:0/210998892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb350197640 con 0x7fb350106560 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.973+0000 7fb354d5e640 1 --2- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 0x7fb350197100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb344009230 tx=0x7fb344003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.973+0000 7fb34dffb640 1 -- 172.21.15.31:0/210998892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb34403f070 con 0x7fb350106560 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.973+0000 7fb34dffb640 1 -- 172.21.15.31:0/210998892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34403bd40 con 0x7fb350106560 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.973+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3501978d0 con 0x7fb350106560 2024-09-16T00:32:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.973+0000 7fb34dffb640 1 -- 172.21.15.31:0/210998892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb344035030 con 0x7fb350106560 2024-09-16T00:32:23.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.973+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35019b9d0 con 0x7fb350106560 2024-09-16T00:32:23.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.974+0000 7fb34dffb640 1 -- 172.21.15.31:0/210998892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb344042070 con 0x7fb350106560 2024-09-16T00:32:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.974+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb350101bd0 con 0x7fb350106560 2024-09-16T00:32:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.974+0000 7fb34dffb640 1 --2- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb340039300 0x7fb34003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.974+0000 7fb34dffb640 1 -- 172.21.15.31:0/210998892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb34406cf10 con 0x7fb350106560 2024-09-16T00:32:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.974+0000 7fb34ffff640 1 --2- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb340039300 0x7fb34003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.975+0000 7fb34ffff640 1 --2- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb340039300 0x7fb34003b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0098c0 tx=0x7fb33c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:23.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:23.979+0000 7fb34dffb640 1 -- 172.21.15.31:0/210998892 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb34403a8e0 con 0x7fb350106560 2024-09-16T00:32:24.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.157+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb3501069c0 con 0x7fb350106560 2024-09-16T00:32:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.334+0000 7fb34dffb640 1 -- 172.21.15.31:0/210998892 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb34403a3c0 con 0x7fb350106560 2024-09-16T00:32:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-16T00:32:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.336+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb340039300 msgr2=0x7fb34003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.336+0000 7fb356fc2640 1 --2- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb340039300 0x7fb34003b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0098c0 tx=0x7fb33c006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.336+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 msgr2=0x7fb350197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.336+0000 7fb356fc2640 1 --2- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 0x7fb350197100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb344009230 tx=0x7fb344003d80 comp rx=0 tx=0).stop 2024-09-16T00:32:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.336+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 shutdown_connections 2024-09-16T00:32:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.336+0000 7fb356fc2640 1 --2- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb340039300 0x7fb34003b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.336+0000 7fb356fc2640 1 --2- 172.21.15.31:0/210998892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb350106560 0x7fb350197100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.336+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 >> 172.21.15.31:0/210998892 conn(0x7fb3500fc1b0 msgr2=0x7fb350069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.336+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 shutdown_connections 2024-09-16T00:32:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:24.337+0000 7fb356fc2640 1 -- 172.21.15.31:0/210998892 wait complete. 2024-09-16T00:32:24.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:24 smithi031 ceph-mon[30956]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:24.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:24 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/210998892' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T00:32:24.888 INFO:tasks.cephadm:Adding mon.a on smithi031 2024-09-16T00:32:24.888 INFO:tasks.cephadm:Adding mon.b on smithi081 2024-09-16T00:32:24.889 INFO:tasks.cephadm:Adding mon.c on smithi102 2024-09-16T00:32:24.889 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c' 2024-09-16T00:32:25.457 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:32:25.501 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:32:25.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:25 smithi031 ceph-mon[30956]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:25.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:25 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/210998892' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T00:32:25.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:25 smithi031 ceph-mon[30956]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:26.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.986+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2808365291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 msgr2=0x7f6e581019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:26.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.986+0000 7f6e5ebf2640 1 --2- 172.21.15.102:0/2808365291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 0x7f6e581019e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009880 tx=0x7f6e5002a5d0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.986+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2808365291 shutdown_connections 2024-09-16T00:32:26.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.986+0000 7f6e5ebf2640 1 --2- 172.21.15.102:0/2808365291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 0x7f6e581019e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.986+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2808365291 >> 172.21.15.102:0/2808365291 conn(0x7f6e580fcd70 msgr2=0x7f6e580ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:26.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.986+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2808365291 shutdown_connections 2024-09-16T00:32:26.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.986+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2808365291 wait complete. 2024-09-16T00:32:26.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.987+0000 7f6e5ebf2640 1 Processor -- start 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.987+0000 7f6e5ebf2640 1 -- start start 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.987+0000 7f6e5ebf2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 0x7f6e58075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.987+0000 7f6e5ebf2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581076f0 con 0x7f6e581015e0 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.987+0000 7f6e5c98e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 0x7f6e58075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.987+0000 7f6e5c98e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 0x7f6e58075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.102:57824/0 (socket says 172.21.15.102:57824) 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.987+0000 7f6e5c98e640 1 -- 172.21.15.102:0/2653741550 learned_addr learned my addr 172.21.15.102:0/2653741550 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.988+0000 7f6e5c98e640 1 -- 172.21.15.102:0/2653741550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e580763f0 con 0x7f6e581015e0 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.988+0000 7f6e5c98e640 1 --2- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 0x7f6e58075eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e50004f60 tx=0x7f6e500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.988+0000 7f6e55ffb640 1 -- 172.21.15.102:0/2653741550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e5003f070 con 0x7f6e581015e0 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.988+0000 7f6e55ffb640 1 -- 172.21.15.102:0/2653741550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e50004d40 con 0x7f6e581015e0 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.988+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e580745e0 con 0x7f6e581015e0 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.988+0000 7f6e55ffb640 1 -- 172.21.15.102:0/2653741550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e50035030 con 0x7f6e581015e0 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.988+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e58074b20 con 0x7f6e581015e0 2024-09-16T00:32:26.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.989+0000 7f6e55ffb640 1 -- 172.21.15.102:0/2653741550 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6e50042070 con 0x7f6e581015e0 2024-09-16T00:32:26.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.989+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e58107e70 con 0x7f6e581015e0 2024-09-16T00:32:26.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.989+0000 7f6e55ffb640 1 --2- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6e440392a0 0x7f6e4403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:26.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.989+0000 7f6e55ffb640 1 -- 172.21.15.102:0/2653741550 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6e5006d1a0 con 0x7f6e581015e0 2024-09-16T00:32:26.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.989+0000 7f6e57fff640 1 --2- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6e440392a0 0x7f6e4403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:26.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.990+0000 7f6e57fff640 1 --2- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6e440392a0 0x7f6e4403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800ad30 tx=0x7f6e480093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:26.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:25.994+0000 7f6e55ffb640 1 -- 172.21.15.102:0/2653741550 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e500303a0 con 0x7f6e581015e0 2024-09-16T00:32:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.182+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}) -- 0x7f6e581901b0 con 0x7f6e440392a0 2024-09-16T00:32:26.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.187+0000 7f6e55ffb640 1 -- 172.21.15.102:0/2653741550 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6e581901b0 con 0x7f6e440392a0 2024-09-16T00:32:26.188 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mon update... 2024-09-16T00:32:26.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6e440392a0 msgr2=0x7f6e4403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:26.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 --2- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6e440392a0 0x7f6e4403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800ad30 tx=0x7f6e480093f0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 msgr2=0x7f6e58075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:26.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 --2- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 0x7f6e58075eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e50004f60 tx=0x7f6e500048f0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 shutdown_connections 2024-09-16T00:32:26.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 --2- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6e440392a0 0x7f6e4403b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 --2- 172.21.15.102:0/2653741550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581015e0 0x7f6e58075eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 >> 172.21.15.102:0/2653741550 conn(0x7f6e580fcd70 msgr2=0x7f6e580fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:26.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 shutdown_connections 2024-09-16T00:32:26.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:26.190+0000 7f6e5ebf2640 1 -- 172.21.15.102:0/2653741550 wait complete. 2024-09-16T00:32:26.613 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:26.613 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:26.613 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:26.613 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:26.613 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:32:26.613 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:26.613 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:26.614 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-16T00:32:26.614 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:26.614 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:26.614 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:26.614 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:26.614 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:26.614 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:26.793 DEBUG:teuthology.orchestra.run.smithi081:mon.b> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T00:32:26.796 DEBUG:teuthology.orchestra.run.smithi102:mon.c> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T00:32:26.799 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:32:26.799 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:32:27.619 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:27 smithi031 ceph-mon[30956]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:27.619 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:27 smithi031 ceph-mon[30956]: Updating smithi102:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:27.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:27 smithi031 ceph-mon[30956]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:27.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:27 smithi031 ceph-mon[30956]: Updating smithi102:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:32:27.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:27 smithi031 ceph-mon[30956]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:27.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:27 smithi031 ceph-mon[30956]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c;count:3 2024-09-16T00:32:27.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:27 smithi031 ceph-mon[30956]: Deploying daemon mon.c on smithi102 2024-09-16T00:32:28.633 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.212+0000 7f9669519640 1 -- 172.21.15.102:0/3345443565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 msgr2=0x7f9664102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.212+0000 7f9669519640 1 --2- 172.21.15.102:0/3345443565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 0x7f9664102ce0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9654009880 tx=0x7f965402a670 comp rx=0 tx=0).stop 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.212+0000 7f9669519640 1 -- 172.21.15.102:0/3345443565 shutdown_connections 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.212+0000 7f9669519640 1 --2- 172.21.15.102:0/3345443565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 0x7f9664102ce0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.212+0000 7f9669519640 1 -- 172.21.15.102:0/3345443565 >> 172.21.15.102:0/3345443565 conn(0x7f96640fc280 msgr2=0x7f96640fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.212+0000 7f9669519640 1 -- 172.21.15.102:0/3345443565 shutdown_connections 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.212+0000 7f9669519640 1 -- 172.21.15.102:0/3345443565 wait complete. 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.213+0000 7f9669519640 1 Processor -- start 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.213+0000 7f9669519640 1 -- start start 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.214+0000 7f9669519640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 0x7f96641972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:29.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.214+0000 7f9669519640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f966410bda0 con 0x7f9664102900 2024-09-16T00:32:29.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.214+0000 7f9662ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 0x7f96641972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:29.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.214+0000 7f9662ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 0x7f96641972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.102:57842/0 (socket says 172.21.15.102:57842) 2024-09-16T00:32:29.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.214+0000 7f9662ffd640 1 -- 172.21.15.102:0/3184816503 learned_addr learned my addr 172.21.15.102:0/3184816503 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:32:29.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.215+0000 7f9662ffd640 1 -- 172.21.15.102:0/3184816503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96641977f0 con 0x7f9664102900 2024-09-16T00:32:29.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.215+0000 7f9662ffd640 1 --2- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 0x7f96641972b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9654009230 tx=0x7f9654003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:29.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.215+0000 7f964bfff640 1 -- 172.21.15.102:0/3184816503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f965403f070 con 0x7f9664102900 2024-09-16T00:32:29.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.215+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9664197a80 con 0x7f9664102900 2024-09-16T00:32:29.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.215+0000 7f964bfff640 1 -- 172.21.15.102:0/3184816503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f965403bc80 con 0x7f9664102900 2024-09-16T00:32:29.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.215+0000 7f964bfff640 1 -- 172.21.15.102:0/3184816503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9654035030 con 0x7f9664102900 2024-09-16T00:32:29.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.215+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9664197ee0 con 0x7f9664102900 2024-09-16T00:32:29.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.216+0000 7f964bfff640 1 -- 172.21.15.102:0/3184816503 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9654042070 con 0x7f9664102900 2024-09-16T00:32:29.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.216+0000 7f964bfff640 1 --2- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f964c0392a0 0x7f964c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:29.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.217+0000 7f964bfff640 1 -- 172.21.15.102:0/3184816503 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f965406cfb0 con 0x7f9664102900 2024-09-16T00:32:29.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.217+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f966406a430 con 0x7f9664102900 2024-09-16T00:32:29.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.217+0000 7f96627fc640 1 --2- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f964c0392a0 0x7f964c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:29.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.217+0000 7f96627fc640 1 --2- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f964c0392a0 0x7f964c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f96580098c0 tx=0x7f9658006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:29.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.222+0000 7f964bfff640 1 -- 172.21.15.102:0/3184816503 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f965404c350 con 0x7f9664102900 2024-09-16T00:32:29.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.493+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9664198610 con 0x7f9664102900 2024-09-16T00:32:29.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.494+0000 7f964bfff640 1 -- 172.21.15.102:0/3184816503 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f96540314c0 con 0x7f9664102900 2024-09-16T00:32:29.496 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:32:29.496 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":1,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","modified":"2024-09-16T00:31:01.292640Z","created":"2024-09-16T00:31:01.292640Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:32:29.496 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 1 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f964c0392a0 msgr2=0x7f964c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 --2- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f964c0392a0 0x7f964c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f96580098c0 tx=0x7f9658006db0 comp rx=0 tx=0).stop 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 msgr2=0x7f96641972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 --2- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 0x7f96641972b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9654009230 tx=0x7f9654003d40 comp rx=0 tx=0).stop 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 shutdown_connections 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 --2- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f964c0392a0 0x7f964c03b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 --2- 172.21.15.102:0/3184816503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9664102900 0x7f96641972b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 >> 172.21.15.102:0/3184816503 conn(0x7f96640fc280 msgr2=0x7f96641056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 shutdown_connections 2024-09-16T00:32:29.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:29.496+0000 7f9669519640 1 -- 172.21.15.102:0/3184816503 wait complete. 2024-09-16T00:32:29.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:29 smithi031 ceph-mon[30956]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:30.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:30 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/3184816503' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:32:31.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:30 smithi102 systemd[1]: Started Ceph mon.c for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:32:31.081 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:32:31.082 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:32:31.333 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:32:36.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: mon.a calling monitor election 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: mon.c calling monitor election 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: monmap epoch 2 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: last_changed 2024-09-16T00:32:30.573134+0000 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: min_mon_release 19 (squid) 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: election_strategy: 1 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: fsmap 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: mgrmap e13: x(active, since 38s) 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: overall HEALTH_OK 2024-09-16T00:32:36.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:36.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:36.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:36.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:36.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: mon.a calling monitor election 2024-09-16T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: mon.c calling monitor election 2024-09-16T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: monmap epoch 2 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: last_changed 2024-09-16T00:32:30.573134+0000 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: min_mon_release 19 (squid) 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: election_strategy: 1 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:32:36.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-16T00:32:36.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: fsmap 2024-09-16T00:32:36.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:36.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: mgrmap e13: x(active, since 38s) 2024-09-16T00:32:36.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: overall HEALTH_OK 2024-09-16T00:32:36.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:36.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:36.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:36.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:36.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:36.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.151+0000 7efd30a11640 1 -- 172.21.15.102:0/1554817260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd18003530 msgr2=0x7efd180059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.151+0000 7efd30a11640 1 --2- 172.21.15.102:0/1554817260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd18003530 0x7efd180059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efd2c1016a0 tx=0x7efd2402c490 comp rx=0 tx=0).stop 2024-09-16T00:32:36.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.151+0000 7efd30a11640 1 -- 172.21.15.102:0/1554817260 shutdown_connections 2024-09-16T00:32:36.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.151+0000 7efd30a11640 1 --2- 172.21.15.102:0/1554817260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd18003530 0x7efd180059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.151+0000 7efd30a11640 1 -- 172.21.15.102:0/1554817260 >> 172.21.15.102:0/1554817260 conn(0x7efd2c0fc220 msgr2=0x7efd2c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:36.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.151+0000 7efd30a11640 1 -- 172.21.15.102:0/1554817260 shutdown_connections 2024-09-16T00:32:36.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.151+0000 7efd30a11640 1 -- 172.21.15.102:0/1554817260 wait complete. 2024-09-16T00:32:36.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.152+0000 7efd30a11640 1 Processor -- start 2024-09-16T00:32:36.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.152+0000 7efd30a11640 1 -- start start 2024-09-16T00:32:36.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.152+0000 7efd30a11640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 0x7efd2c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.152+0000 7efd30a11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c1999c0 0x7efd2c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.152+0000 7efd30a11640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c108670 con 0x7efd2c1999c0 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.152+0000 7efd30a11640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1084f0 con 0x7efd18003530 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd2a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 0x7efd2c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd2a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 0x7efd2c199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41894/0 (socket says 172.21.15.102:41894) 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd2a59c640 1 -- 172.21.15.102:0/875016906 learned_addr learned my addr 172.21.15.102:0/875016906 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd29d9b640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c1999c0 0x7efd2c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd2a59c640 1 -- 172.21.15.102:0/875016906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 msgr2=0x7efd2c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd2a59c640 1 -- 172.21.15.102:0/875016906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 msgr2=0x7efd2c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd2a59c640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 0x7efd2c199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd2a59c640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 0x7efd2c199480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd29d9b640 1 -- 172.21.15.102:0/875016906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 msgr2=0x7efd2c199480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:32:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd29d9b640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 0x7efd2c199480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd29d9b640 1 -- 172.21.15.102:0/875016906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c199fe0 con 0x7efd2c1999c0 2024-09-16T00:32:36.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.153+0000 7efd29d9b640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c1999c0 0x7efd2c19dd50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd1400b3d0 tx=0x7efd1400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:36.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.154+0000 7efd237fe640 1 -- 172.21.15.102:0/875016906 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7efd14011630 con 0x7efd2c1999c0 2024-09-16T00:32:36.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.154+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c19e290 con 0x7efd2c1999c0 2024-09-16T00:32:36.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.154+0000 7efd237fe640 1 -- 172.21.15.102:0/875016906 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd14011cb0 con 0x7efd2c1999c0 2024-09-16T00:32:36.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.154+0000 7efd237fe640 1 -- 172.21.15.102:0/875016906 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7efd14010df0 con 0x7efd2c1999c0 2024-09-16T00:32:36.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.154+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c19e850 con 0x7efd2c1999c0 2024-09-16T00:32:36.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.155+0000 7efd237fe640 1 -- 172.21.15.102:0/875016906 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7efd14018020 con 0x7efd2c1999c0 2024-09-16T00:32:36.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.155+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2c19eb60 con 0x7efd2c1999c0 2024-09-16T00:32:36.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.155+0000 7efd237fe640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efd0c039040 0x7efd0c03b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.155+0000 7efd237fe640 1 -- 172.21.15.102:0/875016906 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efd1404bf80 con 0x7efd2c1999c0 2024-09-16T00:32:36.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.155+0000 7efd2a59c640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efd0c039040 0x7efd0c03b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.156+0000 7efd2a59c640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efd0c039040 0x7efd0c03b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd24012c10 tx=0x7efd240323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:36.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.161+0000 7efd237fe640 1 -- 172.21.15.102:0/875016906 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd1401f020 con 0x7efd2c1999c0 2024-09-16T00:32:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.392+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efd2c19a840 con 0x7efd2c1999c0 2024-09-16T00:32:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.393+0000 7efd237fe640 1 -- 172.21.15.102:0/875016906 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7efd14018310 con 0x7efd2c1999c0 2024-09-16T00:32:36.393 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:32:36.393 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","modified":"2024-09-16T00:32:30.573134Z","created":"2024-09-16T00:31:01.292640Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:32:36.394 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 2 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efd0c039040 msgr2=0x7efd0c03b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efd0c039040 0x7efd0c03b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd24012c10 tx=0x7efd240323a0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c1999c0 msgr2=0x7efd2c19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c1999c0 0x7efd2c19dd50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd1400b3d0 tx=0x7efd1400b890 comp rx=0 tx=0).stop 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 shutdown_connections 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c1999c0 0x7efd2c19dd50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efd0c039040 0x7efd0c03b500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 --2- 172.21.15.102:0/875016906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd18003530 0x7efd2c199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 >> 172.21.15.102:0/875016906 conn(0x7efd2c0fc220 msgr2=0x7efd2c06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 shutdown_connections 2024-09-16T00:32:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:36.395+0000 7efd30a11640 1 -- 172.21.15.102:0/875016906 wait complete. 2024-09-16T00:32:36.745 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:36 smithi102 ceph-mon[35364]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:36.745 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:36 smithi102 ceph-mon[35364]: Deploying daemon mon.b on smithi081 2024-09-16T00:32:36.746 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:36 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/875016906' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:32:36.746 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:36 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:37.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:36 smithi031 ceph-mon[30956]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:37.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:36 smithi031 ceph-mon[30956]: Deploying daemon mon.b on smithi081 2024-09-16T00:32:37.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:36 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/875016906' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:32:37.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:36 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:38.024 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:32:38.024 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:32:38.276 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:32:38.687 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:38 smithi102 ceph-mon[35364]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.687+0000 7fe7e55d2640 1 -- 172.21.15.102:0/3388040850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0106630 msgr2=0x7fe7e0106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.687+0000 7fe7e55d2640 1 --2- 172.21.15.102:0/3388040850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0106630 0x7fe7e0106a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009880 tx=0x7fe7d002a5d0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.687+0000 7fe7e55d2640 1 -- 172.21.15.102:0/3388040850 shutdown_connections 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.687+0000 7fe7e55d2640 1 --2- 172.21.15.102:0/3388040850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0106630 0x7fe7e0106a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.687+0000 7fe7e55d2640 1 -- 172.21.15.102:0/3388040850 >> 172.21.15.102:0/3388040850 conn(0x7fe7e00fc220 msgr2=0x7fe7e00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.687+0000 7fe7e55d2640 1 -- 172.21.15.102:0/3388040850 shutdown_connections 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.687+0000 7fe7e55d2640 1 -- 172.21.15.102:0/3388040850 wait complete. 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7e55d2640 1 Processor -- start 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7e55d2640 1 -- start start 2024-09-16T00:32:38.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7e55d2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 0x7fe7e0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7e55d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0198750 0x7fe7e0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7e55d2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e010ac60 con 0x7fe7e0198750 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7e55d2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e010aae0 con 0x7fe7e0106630 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 0x7fe7e0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 0x7fe7e0198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41896/0 (socket says 172.21.15.102:41896) 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7deffd640 1 -- 172.21.15.102:0/2210494404 learned_addr learned my addr 172.21.15.102:0/2210494404 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7deffd640 1 -- 172.21.15.102:0/2210494404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 msgr2=0x7fe7e0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7deffd640 1 -- 172.21.15.102:0/2210494404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 msgr2=0x7fe7e0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7deffd640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 0x7fe7e0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7de7fc640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0198750 0x7fe7e0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.688+0000 7fe7deffd640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 0x7fe7e0198210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:32:38.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.689+0000 7fe7de7fc640 1 -- 172.21.15.102:0/2210494404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 msgr2=0x7fe7e0198210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.689+0000 7fe7de7fc640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 0x7fe7e0198210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.689+0000 7fe7de7fc640 1 -- 172.21.15.102:0/2210494404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e01a1280 con 0x7fe7e0198750 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.689+0000 7fe7de7fc640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0198750 0x7fe7e0198bb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b3d0 tx=0x7fe7d400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.689+0000 7fe7c7fff640 1 -- 172.21.15.102:0/2210494404 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe7d4011630 con 0x7fe7e0198750 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.689+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e01a1450 con 0x7fe7e0198750 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.689+0000 7fe7c7fff640 1 -- 172.21.15.102:0/2210494404 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d4011cb0 con 0x7fe7e0198750 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.689+0000 7fe7c7fff640 1 -- 172.21.15.102:0/2210494404 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe7d4010df0 con 0x7fe7e0198750 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.689+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e01a19e0 con 0x7fe7e0198750 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.690+0000 7fe7c7fff640 1 -- 172.21.15.102:0/2210494404 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fe7d4018020 con 0x7fe7e0198750 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.690+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e0101ca0 con 0x7fe7e0198750 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.690+0000 7fe7c7fff640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe7c8039040 0x7fe7c803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.690+0000 7fe7c7fff640 1 -- 172.21.15.102:0/2210494404 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe7d404c100 con 0x7fe7e0198750 2024-09-16T00:32:38.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.690+0000 7fe7deffd640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe7c8039040 0x7fe7c803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:38.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.691+0000 7fe7deffd640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe7c8039040 0x7fe7c803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00096c0 tx=0x7fe7d0005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:38.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.695+0000 7fe7c7fff640 1 -- 172.21.15.102:0/2210494404 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7d401f020 con 0x7fe7e0198750 2024-09-16T00:32:38.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.920+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe7e01a1cf0 con 0x7fe7e0198750 2024-09-16T00:32:38.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.921+0000 7fe7c7fff640 1 -- 172.21.15.102:0/2210494404 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fe7d4021300 con 0x7fe7e0198750 2024-09-16T00:32:38.922 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:32:38.922 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","modified":"2024-09-16T00:32:30.573134Z","created":"2024-09-16T00:31:01.292640Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:32:38.922 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 2 2024-09-16T00:32:38.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe7c8039040 msgr2=0x7fe7c803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:38.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe7c8039040 0x7fe7c803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00096c0 tx=0x7fe7d0005a40 comp rx=0 tx=0).stop 2024-09-16T00:32:38.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0198750 msgr2=0x7fe7e0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:38.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0198750 0x7fe7e0198bb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b3d0 tx=0x7fe7d400b890 comp rx=0 tx=0).stop 2024-09-16T00:32:38.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 shutdown_connections 2024-09-16T00:32:38.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0198750 0x7fe7e0198bb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe7c8039040 0x7fe7c803b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 --2- 172.21.15.102:0/2210494404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7e0106630 0x7fe7e0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 >> 172.21.15.102:0/2210494404 conn(0x7fe7e00fc220 msgr2=0x7fe7e00fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:38.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 shutdown_connections 2024-09-16T00:32:38.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:38.923+0000 7fe7e55d2640 1 -- 172.21.15.102:0/2210494404 wait complete. 2024-09-16T00:32:39.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:38 smithi031 ceph-mon[30956]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:39.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: load: jerasure load: lrc 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: RocksDB version: 7.9.2 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Git sha 0 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: DB SUMMARY 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: DB Session ID: 9BSBA6AO388T6G7J8R8R 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:32:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.env: 0x555c794d5940 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.info_log: 0x555c79f896e0 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.statistics: (nil) 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.use_fsync: 0 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:32:39.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.db_log_dir: 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.wal_dir: 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:32:40.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:32:40.001 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:32:40.002 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:32:40.002 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:32:40.002 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:32:40.002 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.write_buffer_manager: 0x555c79f98dc0 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.unordered_write: 0 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:32:40.003 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.row_cache: None 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.wal_filter: None 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.wal_compression: 0 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_open_files: -1 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:32:40.006 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Compression algorithms supported: 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: kZSTD supported: 0 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:32:40.007 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.merge_operator: 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_filter: None 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555c79f89320) 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:32:40.008 INFO:journalctl@ceph.mon.b.smithi081.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: index_type: 0 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: data_block_index_type: 0 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: index_shortening: 1 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: checksum: 4 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: no_block_cache: 0 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache: 0x555c79f7d1f0 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache_options: 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: capacity : 536870912 2024-09-16T00:32:40.009 INFO:journalctl@ceph.mon.b.smithi081.stdout: num_shard_bits : 4 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: strict_capacity_limit : 0 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache_compressed: (nil) 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: persistent_cache: (nil) 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_size: 4096 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_size_deviation: 10 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_restart_interval: 16 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: index_block_restart_interval: 1 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: metadata_block_size: 4096 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: partition_filters: 0 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: use_delta_encoding: 1 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: filter_policy: bloomfilter 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: whole_key_filtering: 1 2024-09-16T00:32:40.010 INFO:journalctl@ceph.mon.b.smithi081.stdout: verify_compression: 0 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout: format_version: 5 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout: enable_index_compression: 1 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_align: 0 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout: max_auto_readahead_size: 262144 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout: prepopulate_block_cache: 0 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout: initial_auto_readahead_size: 8192 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression: NoCompression 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:32:40.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.num_levels: 7 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:32:40.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:32:40.013 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:32:40.014 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:32:40.015 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:32:40.016 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.ttl: 2592000 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:32:40.017 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cfb9d25f-9680-479b-9566-a266432695f4 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446759569730, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446759570137, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726446759, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cfb9d25f-9680-479b-9566-a266432695f4", "db_session_id": "9BSBA6AO388T6G7J8R8R", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446759570231, "job": 1, "event": "recovery_finished"} 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555c79faae00 2024-09-16T00:32:40.018 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: DB pointer 0x555c7a0a6000 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** DB Stats ** 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** Compaction Stats [default] ** 2024-09-16T00:32:40.019 INFO:journalctl@ceph.mon.b.smithi081.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** Compaction Stats [default] ** 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-09-16T00:32:40.020 INFO:journalctl@ceph.mon.b.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: Block cache BinnedLRUCache@0x555c79f7d1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-09-16T00:32:40.021 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: using public_addr v2:172.21.15.81:0/0 -> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: starting mon.b rank -1 at public addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] at bind addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(???) e0 preinit fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).mds e1 new map 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout: e1 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout: btime 2024-09-16T00:31:04:619302+0000 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout: legacy client fscid: -1 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-09-16T00:32:40.022 INFO:journalctl@ceph.mon.b.smithi081.stdout: No filesystems configured 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/210998892' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T00:32:40.023 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/210998892' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.024 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: Updating smithi102:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: Updating smithi102:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c;count:3 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: Deploying daemon mon.c on smithi102 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/3184816503' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:32:40.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.a calling monitor election 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.c calling monitor election 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:32:40.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: monmap epoch 2 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: last_changed 2024-09-16T00:32:30.573134+0000 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: min_mon_release 19 (squid) 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: election_strategy: 1 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: fsmap 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mgrmap e13: x(active, since 38s) 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: overall HEALTH_OK 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.027 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: Deploying daemon mon.b on smithi081 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/875016906' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 ceph-mon[35539]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 bash[35520]: ad386f878d0ac94fe642ebce62dade189c2ec2e22daabfc2cdeda80a6d905e22 2024-09-16T00:32:40.028 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:39 smithi081 systemd[1]: Started Ceph mon.b for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:32:40.447 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:32:40.448 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:32:40.711 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:32:44.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.745+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2881725721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 msgr2=0x7f67980059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:44.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.745+0000 7f67b4e0a640 1 --2- 172.21.15.102:0/2881725721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 0x7f67980059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67b01016a0 tx=0x7f67a002c3f0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.745+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2881725721 shutdown_connections 2024-09-16T00:32:44.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.745+0000 7f67b4e0a640 1 --2- 172.21.15.102:0/2881725721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 0x7f67980059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.745+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2881725721 >> 172.21.15.102:0/2881725721 conn(0x7f67b00fc220 msgr2=0x7f67b00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:44.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.746+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2881725721 shutdown_connections 2024-09-16T00:32:44.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.746+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2881725721 wait complete. 2024-09-16T00:32:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.746+0000 7f67b4e0a640 1 Processor -- start 2024-09-16T00:32:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67b4e0a640 1 -- start start 2024-09-16T00:32:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67b4e0a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 0x7f67b01972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67b4e0a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 0x7f67b019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67b4e0a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67b019c0e0 0x7f67b019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67b4e0a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0108670 con 0x7f6798003530 2024-09-16T00:32:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67b4e0a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01084f0 con 0x7f67b019c0e0 2024-09-16T00:32:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67b4e0a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01087f0 con 0x7f67b0197810 2024-09-16T00:32:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67add9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 0x7f67b019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 0x7f67b01972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67add9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 0x7f67b019bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48596/0 (socket says 172.21.15.102:48596) 2024-09-16T00:32:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 0x7f67b01972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.102:58148/0 (socket says 172.21.15.102:58148) 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67add9b640 1 -- 172.21.15.102:0/2824896462 learned_addr learned my addr 172.21.15.102:0/2824896462 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67add9b640 1 -- 172.21.15.102:0/2824896462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 msgr2=0x7f67b019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67add9b640 1 -- 172.21.15.102:0/2824896462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 msgr2=0x7f67b019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67add9b640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 0x7f67b019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67add9b640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 0x7f67b019bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67aed9d640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67b019c0e0 0x7f67b019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67ae59c640 1 -- 172.21.15.102:0/2824896462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 msgr2=0x7f67b019bba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67ae59c640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 0x7f67b019bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67ae59c640 1 -- 172.21.15.102:0/2824896462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67b019c0e0 msgr2=0x7f67b019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67ae59c640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67b019c0e0 0x7f67b019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67ae59c640 1 -- 172.21.15.102:0/2824896462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b01a3e50 con 0x7f6798003530 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.747+0000 7f67aed9d640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67b019c0e0 0x7f67b019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.748+0000 7f67ae59c640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 0x7f67b01972d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67a0036040 tx=0x7f67a0004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:44.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.748+0000 7f679f7fe640 1 -- 172.21.15.102:0/2824896462 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0041030 con 0x7f6798003530 2024-09-16T00:32:44.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.748+0000 7f679f7fe640 1 -- 172.21.15.102:0/2824896462 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a0004c50 con 0x7f6798003530 2024-09-16T00:32:44.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.748+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b01a4020 con 0x7f6798003530 2024-09-16T00:32:44.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.748+0000 7f679f7fe640 1 -- 172.21.15.102:0/2824896462 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a003d6a0 con 0x7f6798003530 2024-09-16T00:32:44.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.748+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b01a4600 con 0x7f6798003530 2024-09-16T00:32:44.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.749+0000 7f679f7fe640 1 -- 172.21.15.102:0/2824896462 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f67a0044070 con 0x7f6798003530 2024-09-16T00:32:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.749+0000 7f679f7fe640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6790039730 0x7f679003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.749+0000 7f679f7fe640 1 -- 172.21.15.102:0/2824896462 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f67a006f220 con 0x7f6798003530 2024-09-16T00:32:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.749+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b01097a0 con 0x7f6798003530 2024-09-16T00:32:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.749+0000 7f67add9b640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6790039730 0x7f679003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.750+0000 7f67add9b640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6790039730 0x7f679003bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009100 tx=0x7f67a4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.755+0000 7f679f7fe640 1 -- 172.21.15.102:0/2824896462 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67a004b070 con 0x7f6798003530 2024-09-16T00:32:44.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.997+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f67b01a4900 con 0x7f6798003530 2024-09-16T00:32:44.998 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:44.998 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:32:44.998 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: mon.a calling monitor election 2024-09-16T00:32:44.998 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: mon.c calling monitor election 2024-09-16T00:32:44.998 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:44.998 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:44.998 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:44.998 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: monmap epoch 3 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: last_changed 2024-09-16T00:32:39.595443+0000 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: min_mon_release 19 (squid) 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: election_strategy: 1 2024-09-16T00:32:44.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: fsmap 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: mgrmap e13: x(active, since 47s) 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: overall HEALTH_OK 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:45.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:45.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:44.998+0000 7f679f7fe640 1 -- 172.21.15.102:0/2824896462 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f67a0042270 con 0x7f6798003530 2024-09-16T00:32:45.001 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:32:45.001 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":3,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","modified":"2024-09-16T00:32:39.595443Z","created":"2024-09-16T00:31:01.292640Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 3 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6790039730 msgr2=0x7f679003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6790039730 0x7f679003bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009100 tx=0x7f67a4008040 comp rx=0 tx=0).stop 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 msgr2=0x7f67b01972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 0x7f67b01972d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67a0036040 tx=0x7f67a0004850 comp rx=0 tx=0).stop 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 shutdown_connections 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67b019c0e0 0x7f67b019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6790039730 0x7f679003bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b0197810 0x7f67b019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 --2- 172.21.15.102:0/2824896462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6798003530 0x7f67b01972d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 >> 172.21.15.102:0/2824896462 conn(0x7f67b00fc220 msgr2=0x7f67b0105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:45.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 shutdown_connections 2024-09-16T00:32:45.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:45.000+0000 7f67b4e0a640 1 -- 172.21.15.102:0/2824896462 wait complete. 2024-09-16T00:32:45.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:45.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:32:45.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: mon.a calling monitor election 2024-09-16T00:32:45.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: mon.c calling monitor election 2024-09-16T00:32:45.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:45.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:45.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:45.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:45.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:45.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:45.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:45.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:45.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:45.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:32:45.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: monmap epoch 3 2024-09-16T00:32:45.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:45.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: last_changed 2024-09-16T00:32:39.595443+0000 2024-09-16T00:32:45.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:32:45.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: min_mon_release 19 (squid) 2024-09-16T00:32:45.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: election_strategy: 1 2024-09-16T00:32:45.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:32:45.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-16T00:32:45.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-16T00:32:45.129 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: fsmap 2024-09-16T00:32:45.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:45.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: mgrmap e13: x(active, since 47s) 2024-09-16T00:32:45.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: overall HEALTH_OK 2024-09-16T00:32:45.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:45.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:45.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:45.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:45.575 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T00:32:45.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T00:32:45.844 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:45.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:45.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:45 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/2824896462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:32:45.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:45.878 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:45.878 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:45 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/2824896462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:32:45.878 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.257+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3091348135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc01002c0 msgr2=0x7f5dc01006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.257+0000 7f5dc77e9640 1 --2- 172.21.15.31:0/3091348135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc01002c0 0x7f5dc01006a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009810 tx=0x7f5dbc02a540 comp rx=0 tx=0).stop 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.257+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3091348135 shutdown_connections 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.257+0000 7f5dc77e9640 1 --2- 172.21.15.31:0/3091348135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc01002c0 0x7f5dc01006a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.257+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3091348135 >> 172.21.15.31:0/3091348135 conn(0x7f5dc0077310 msgr2=0x7f5dc00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.257+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3091348135 shutdown_connections 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.257+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3091348135 wait complete. 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.257+0000 7f5dc77e9640 1 Processor -- start 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc77e9640 1 -- start start 2024-09-16T00:32:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc77e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc01002c0 0x7f5dc0199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc77e9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dc0199aa0 0x7f5dc019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc77e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc019e370 0x7f5dc019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc77e9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc010de80 con 0x7f5dc019e370 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc77e9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc010dd00 con 0x7f5dc01002c0 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc77e9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc010e000 con 0x7f5dc0199aa0 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5d86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc019e370 0x7f5dc019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5d86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc019e370 0x7f5dc019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55550/0 (socket says 172.21.15.31:55550) 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5d86640 1 -- 172.21.15.31:0/3213550811 learned_addr learned my addr 172.21.15.31:0/3213550811 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc4d84640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dc0199aa0 0x7f5dc019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5585640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc01002c0 0x7f5dc0199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5d86640 1 -- 172.21.15.31:0/3213550811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dc0199aa0 msgr2=0x7f5dc019de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5d86640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dc0199aa0 0x7f5dc019de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5585640 1 -- 172.21.15.31:0/3213550811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc01002c0 msgr2=0x7f5dc0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5585640 1 -- 172.21.15.31:0/3213550811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc01002c0 msgr2=0x7f5dc0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5585640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc01002c0 0x7f5dc0199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5d86640 1 -- 172.21.15.31:0/3213550811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc01002c0 msgr2=0x7f5dc0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5585640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc01002c0 0x7f5dc0199560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5d86640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc01002c0 0x7f5dc0199560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.258+0000 7f5dc5d86640 1 -- 172.21.15.31:0/3213550811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc01a60e0 con 0x7f5dc019e370 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.259+0000 7f5dc5d86640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc019e370 0x7f5dc019e820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5db800ba50 tx=0x7f5db800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.259+0000 7f5db67fc640 1 -- 172.21.15.31:0/3213550811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8019070 con 0x7f5dc019e370 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.259+0000 7f5db67fc640 1 -- 172.21.15.31:0/3213550811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db800cbe0 con 0x7f5dc019e370 2024-09-16T00:32:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.259+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc01a6310 con 0x7f5dc019e370 2024-09-16T00:32:46.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.259+0000 7f5db67fc640 1 -- 172.21.15.31:0/3213550811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db80076a0 con 0x7f5dc019e370 2024-09-16T00:32:46.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.260+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc01a6870 con 0x7f5dc019e370 2024-09-16T00:32:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.260+0000 7f5db67fc640 1 -- 172.21.15.31:0/3213550811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f5db801c070 con 0x7f5dc019e370 2024-09-16T00:32:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.260+0000 7f5db67fc640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5dac039730 0x7f5dac03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.260+0000 7f5db67fc640 1 -- 172.21.15.31:0/3213550811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5db8051660 con 0x7f5dc019e370 2024-09-16T00:32:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.260+0000 7f5dc5585640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5dac039730 0x7f5dac03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.261+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc0069f90 con 0x7f5dc019e370 2024-09-16T00:32:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.261+0000 7f5dc5585640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5dac039730 0x7f5dac03bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc005e60 tx=0x7f5dbc02aa40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.265+0000 7f5db67fc640 1 -- 172.21.15.31:0/3213550811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5db800fe60 con 0x7f5dc019e370 2024-09-16T00:32:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.450+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f5dc01a6b50 con 0x7f5dc019e370 2024-09-16T00:32:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.451+0000 7f5db67fc640 1 -- 172.21.15.31:0/3213550811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f5db8012990 con 0x7f5dc019e370 2024-09-16T00:32:46.452 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:46.453 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-16T00:32:46.453 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:46.453 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] 2024-09-16T00:32:46.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5dac039730 msgr2=0x7f5dac03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5dac039730 0x7f5dac03bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc005e60 tx=0x7f5dbc02aa40 comp rx=0 tx=0).stop 2024-09-16T00:32:46.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc019e370 msgr2=0x7f5dc019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc019e370 0x7f5dc019e820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5db800ba50 tx=0x7f5db800bf10 comp rx=0 tx=0).stop 2024-09-16T00:32:46.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 shutdown_connections 2024-09-16T00:32:46.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dc019e370 0x7f5dc019e820 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5dac039730 0x7f5dac03bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dc0199aa0 0x7f5dc019de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 --2- 172.21.15.31:0/3213550811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc01002c0 0x7f5dc0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 >> 172.21.15.31:0/3213550811 conn(0x7f5dc0077310 msgr2=0x7f5dc01088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:46.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 shutdown_connections 2024-09-16T00:32:46.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:46.453+0000 7f5dc77e9640 1 -- 172.21.15.31:0/3213550811 wait complete. 2024-09-16T00:32:46.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:46.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:32:46.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: mon.a calling monitor election 2024-09-16T00:32:46.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: mon.c calling monitor election 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: monmap epoch 3 2024-09-16T00:32:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: last_changed 2024-09-16T00:32:39.595443+0000 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: min_mon_release 19 (squid) 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: election_strategy: 1 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: fsmap 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: mgrmap e13: x(active, since 47s) 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: overall HEALTH_OK 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:46.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:47.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/2824896462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:32:47.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:47.286 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T00:32:47.286 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:32:47.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:32:47.315 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:32:47.316 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:47.382 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:32:47.382 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:32:47.410 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:32:47.410 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:47.475 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:32:47.475 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:32:47.505 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:32:47.505 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:32:47.572 INFO:tasks.cephadm:Adding mgr.x on smithi031 2024-09-16T00:32:47.573 INFO:tasks.cephadm:Adding mgr.y on smithi081 2024-09-16T00:32:47.573 INFO:tasks.cephadm:Adding mgr.z on smithi102 2024-09-16T00:32:47.573 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi031=x;smithi081=y;smithi102=z' 2024-09-16T00:32:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: mon.b calling monitor election 2024-09-16T00:32:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: Updating smithi102:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: Updating smithi081:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: Updating smithi031:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: mon.b calling monitor election 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: mon.c calling monitor election 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: mon.a calling monitor election 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: monmap epoch 3 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: last_changed 2024-09-16T00:32:39.595443+0000 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: min_mon_release 19 (squid) 2024-09-16T00:32:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: election_strategy: 1 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: fsmap 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: mgrmap e13: x(active, since 49s) 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: overall HEALTH_OK 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:32:47.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:47.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:32:47.751 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:47.751 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:47 smithi081 ceph-mon[35539]: Reconfiguring daemon mon.a on smithi031 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: mon.b calling monitor election 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: Updating smithi102:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: Updating smithi081:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: Updating smithi031:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: mon.b calling monitor election 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: mon.c calling monitor election 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: mon.a calling monitor election 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: monmap epoch 3 2024-09-16T00:32:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: last_changed 2024-09-16T00:32:39.595443+0000 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: min_mon_release 19 (squid) 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: election_strategy: 1 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: fsmap 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: mgrmap e13: x(active, since 49s) 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: overall HEALTH_OK 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:32:47.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:47.803 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:47 smithi102 ceph-mon[35364]: Reconfiguring daemon mon.a on smithi031 2024-09-16T00:32:47.867 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:32:47.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: mon.b calling monitor election 2024-09-16T00:32:47.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: Updating smithi102:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:47.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: Updating smithi081:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:47.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: Updating smithi031:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:32:47.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: mon.b calling monitor election 2024-09-16T00:32:47.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: mon.c calling monitor election 2024-09-16T00:32:47.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: mon.a calling monitor election 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: monmap epoch 3 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: last_changed 2024-09-16T00:32:39.595443+0000 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: created 2024-09-16T00:31:01.292640+0000 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: min_mon_release 19 (squid) 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: election_strategy: 1 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: fsmap 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: mgrmap e13: x(active, since 49s) 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: overall HEALTH_OK 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:47.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:32:47.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:47.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:47 smithi031 ceph-mon[30956]: Reconfiguring daemon mon.a on smithi031 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.222+0000 7f61c9aac640 1 -- 172.21.15.102:0/180303358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 msgr2=0x7f61c41009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.222+0000 7f61c9aac640 1 --2- 172.21.15.102:0/180303358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 0x7f61c41009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009880 tx=0x7f61b402a5d0 comp rx=0 tx=0).stop 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.222+0000 7f61c9aac640 1 -- 172.21.15.102:0/180303358 shutdown_connections 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.222+0000 7f61c9aac640 1 --2- 172.21.15.102:0/180303358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 0x7f61c41009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.222+0000 7f61c9aac640 1 -- 172.21.15.102:0/180303358 >> 172.21.15.102:0/180303358 conn(0x7f61c40fbda0 msgr2=0x7f61c40fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.222+0000 7f61c9aac640 1 -- 172.21.15.102:0/180303358 shutdown_connections 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.222+0000 7f61c9aac640 1 -- 172.21.15.102:0/180303358 wait complete. 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c9aac640 1 Processor -- start 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c9aac640 1 -- start start 2024-09-16T00:32:48.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c9aac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 0x7f61c40fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c9aac640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61c40fc340 0x7f61c40fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c9aac640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c40fcd00 0x7f61c41a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c9aac640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41066e0 con 0x7f61c41005f0 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c9aac640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4106560 con 0x7f61c40fc340 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c9aac640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4106860 con 0x7f61c40fcd00 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 0x7f61c40fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c40fcd00 0x7f61c41a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 0x7f61c40fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.102:58170/0 (socket says 172.21.15.102:58170) 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.223+0000 7f61c37fe640 1 -- 172.21.15.102:0/721472724 learned_addr learned my addr 172.21.15.102:0/721472724 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c2ffd640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61c40fc340 0x7f61c40fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c37fe640 1 -- 172.21.15.102:0/721472724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c40fcd00 msgr2=0x7f61c41a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:48.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c37fe640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c40fcd00 0x7f61c41a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:48.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c37fe640 1 -- 172.21.15.102:0/721472724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61c40fc340 msgr2=0x7f61c40fc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:48.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c37fe640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61c40fc340 0x7f61c40fc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:48.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c37fe640 1 -- 172.21.15.102:0/721472724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c41a1620 con 0x7f61c41005f0 2024-09-16T00:32:48.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c37fe640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 0x7f61c40fdcf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61b4031610 tx=0x7f61b40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:48.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c0ff9640 1 -- 172.21.15.102:0/721472724 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b403f030 con 0x7f61c41005f0 2024-09-16T00:32:48.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c0ff9640 1 -- 172.21.15.102:0/721472724 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b4003ce0 con 0x7f61c41005f0 2024-09-16T00:32:48.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c41a18b0 con 0x7f61c41005f0 2024-09-16T00:32:48.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c0ff9640 1 -- 172.21.15.102:0/721472724 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4035070 con 0x7f61c41005f0 2024-09-16T00:32:48.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.224+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c41a1e60 con 0x7f61c41005f0 2024-09-16T00:32:48.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.225+0000 7f61c0ff9640 1 -- 172.21.15.102:0/721472724 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f61b4042030 con 0x7f61c41005f0 2024-09-16T00:32:48.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.226+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c41a2030 con 0x7f61c41005f0 2024-09-16T00:32:48.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.226+0000 7f61c0ff9640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61ac039730 0x7f61ac03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:48.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.226+0000 7f61c0ff9640 1 -- 172.21.15.102:0/721472724 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f61b406cd60 con 0x7f61c41005f0 2024-09-16T00:32:48.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.226+0000 7f61c2ffd640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61ac039730 0x7f61ac03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:48.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.226+0000 7f61c2ffd640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61ac039730 0x7f61ac03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61c40fda70 tx=0x7f61b8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:48.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.230+0000 7f61c0ff9640 1 -- 172.21.15.102:0/721472724 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61b403ac70 con 0x7f61c41005f0 2024-09-16T00:32:48.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.410+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi081=y;smithi102=z", "target": ["mon-mgr", ""]}) -- 0x7f61c4062630 con 0x7f61ac039730 2024-09-16T00:32:48.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.416+0000 7f61c0ff9640 1 -- 172.21.15.102:0/721472724 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f61c4062630 con 0x7f61ac039730 2024-09-16T00:32:48.417 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mgr update... 2024-09-16T00:32:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61ac039730 msgr2=0x7f61ac03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61ac039730 0x7f61ac03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61c40fda70 tx=0x7f61b8009190 comp rx=0 tx=0).stop 2024-09-16T00:32:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 msgr2=0x7f61c40fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 0x7f61c40fdcf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61b4031610 tx=0x7f61b40043c0 comp rx=0 tx=0).stop 2024-09-16T00:32:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 shutdown_connections 2024-09-16T00:32:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61ac039730 0x7f61ac03bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c40fcd00 0x7f61c41a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:48.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61c40fc340 0x7f61c40fc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:48.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 --2- 172.21.15.102:0/721472724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c41005f0 0x7f61c40fdcf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:48.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.418+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 >> 172.21.15.102:0/721472724 conn(0x7f61c40fbda0 msgr2=0x7f61c406acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:48.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.419+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 shutdown_connections 2024-09-16T00:32:48.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:32:48.419+0000 7f61c9aac640 1 -- 172.21.15.102:0/721472724 wait complete. 2024-09-16T00:32:48.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:48.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:48.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:32:48.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:48.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:32:48.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:48.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: Reconfiguring daemon mon.b on smithi081 2024-09-16T00:32:48.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi081=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:48.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: Saving service mgr spec with placement smithi031=x;smithi081=y;smithi102=z;count:3 2024-09-16T00:32:48.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: Reconfiguring daemon mon.b on smithi081 2024-09-16T00:32:48.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi081=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:48.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: Saving service mgr spec with placement smithi031=x;smithi081=y;smithi102=z;count:3 2024-09-16T00:32:48.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.961 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:48.961 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:32:48.961 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.961 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.961 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.961 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:32:48.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:48.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:32:48.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:48.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: Reconfiguring daemon mon.b on smithi081 2024-09-16T00:32:48.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi081=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:48.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: Saving service mgr spec with placement smithi031=x;smithi081=y;smithi102=z;count:3 2024-09-16T00:32:48.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:48.989 DEBUG:teuthology.orchestra.run.smithi081:mgr.y> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T00:32:48.991 DEBUG:teuthology.orchestra.run.smithi102:mgr.z> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.z.service 2024-09-16T00:32:48.994 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T00:32:48.994 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:32:48.994 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:32:49.014 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:32:49.014 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-16T00:32:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:32:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-16T00:32:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 00:32:06.288656628 +0000 2024-09-16T00:32:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 00:27:46.624668851 +0000 2024-09-16T00:32:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 00:27:46.624668851 +0000 2024-09-16T00:32:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 00:27:46.624668851 +0000 2024-09-16T00:32:49.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:32:49.146 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T00:32:49.146 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T00:32:49.146 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000219539 s, 2.3 MB/s 2024-09-16T00:32:49.147 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:32:49.208 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-16T00:32:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:32:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:49.266 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T00:32:49.267 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:49.267 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:49.267 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 00:32:06.315656184 +0000 2024-09-16T00:32:49.267 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 00:27:46.958663377 +0000 2024-09-16T00:32:49.267 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 00:27:46.958663377 +0000 2024-09-16T00:32:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 00:27:46.958663377 +0000 2024-09-16T00:32:49.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:32:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T00:32:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T00:32:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000209784 s, 2.4 MB/s 2024-09-16T00:32:49.336 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:32:49.395 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-16T00:32:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:32:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:49.453 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-16T00:32:49.453 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:49.453 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:49.453 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 00:32:06.343655724 +0000 2024-09-16T00:32:49.453 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 00:27:47.293657887 +0000 2024-09-16T00:32:49.453 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 00:27:47.293657887 +0000 2024-09-16T00:32:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 00:27:47.293657887 +0000 2024-09-16T00:32:49.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:32:49.518 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T00:32:49.519 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T00:32:49.519 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000209355 s, 2.4 MB/s 2024-09-16T00:32:49.520 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:32:49.577 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-16T00:32:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:32:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-16T00:32:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 00:32:06.370655281 +0000 2024-09-16T00:32:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 00:27:47.626652429 +0000 2024-09-16T00:32:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 00:27:47.626652429 +0000 2024-09-16T00:32:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 00:27:47.626652429 +0000 2024-09-16T00:32:49.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:32:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T00:32:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T00:32:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000225387 s, 2.3 MB/s 2024-09-16T00:32:49.702 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:32:49.759 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-16T00:32:49.759 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:32:49.776 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:32:49.777 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-09-16T00:32:49.831 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:32:49.831 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:49.832 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T00:32:49.832 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:49.832 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:49.832 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-16 00:32:39.010524228 +0000 2024-09-16T00:32:49.832 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-16 00:27:56.292734990 +0000 2024-09-16T00:32:49.832 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-16 00:27:56.292734990 +0000 2024-09-16T00:32:49.832 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-16 00:27:56.292734990 +0000 2024-09-16T00:32:49.832 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:32:49.896 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-16T00:32:49.896 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-16T00:32:49.896 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000191786 s, 2.7 MB/s 2024-09-16T00:32:49.897 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:32:49.953 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-09-16T00:32:50.008 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:32:50.008 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:50.009 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T00:32:50.009 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:50.009 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:50.009 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-16 00:32:39.010524228 +0000 2024-09-16T00:32:50.009 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-16 00:27:56.623729179 +0000 2024-09-16T00:32:50.009 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-16 00:27:56.623729179 +0000 2024-09-16T00:32:50.009 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-16 00:27:56.623729179 +0000 2024-09-16T00:32:50.009 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:32:50.072 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-16T00:32:50.072 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-16T00:32:50.072 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000187744 s, 2.7 MB/s 2024-09-16T00:32:50.073 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:32:50.129 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-09-16T00:32:50.184 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:32:50.184 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:50.185 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-16T00:32:50.185 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:50.185 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:50.185 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-16 00:32:39.010524228 +0000 2024-09-16T00:32:50.185 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-16 00:27:56.961723246 +0000 2024-09-16T00:32:50.185 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-16 00:27:56.961723246 +0000 2024-09-16T00:32:50.185 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-16 00:27:56.961723246 +0000 2024-09-16T00:32:50.185 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:32:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-16T00:32:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-16T00:32:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000170364 s, 3.0 MB/s 2024-09-16T00:32:50.248 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:32:50.304 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-09-16T00:32:50.359 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:32:50.359 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:50.359 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-16T00:32:50.359 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:50.359 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:50.359 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-16 00:32:39.011524210 +0000 2024-09-16T00:32:50.360 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-16 00:27:57.281717628 +0000 2024-09-16T00:32:50.360 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-16 00:27:57.281717628 +0000 2024-09-16T00:32:50.360 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-16 00:27:57.281717628 +0000 2024-09-16T00:32:50.360 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:32:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-16T00:32:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-16T00:32:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000180291 s, 2.8 MB/s 2024-09-16T00:32:50.423 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:32:50.479 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T00:32:50.479 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:32:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:50 smithi081 ceph-mon[35539]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:32:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:32:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:50 smithi081 ceph-mon[35539]: Reconfiguring daemon mon.c on smithi102 2024-09-16T00:32:50.498 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:32:50.498 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-09-16T00:32:50.556 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:32:50.556 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:50.556 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T00:32:50.556 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:50.556 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:50.556 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-16 00:32:29.583989909 +0000 2024-09-16T00:32:50.556 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-16 00:27:59.974759136 +0000 2024-09-16T00:32:50.556 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-16 00:27:59.974759136 +0000 2024-09-16T00:32:50.556 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-16 00:27:59.974759136 +0000 2024-09-16T00:32:50.557 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:32:50.622 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-16T00:32:50.622 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-16T00:32:50.622 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000308205 s, 1.7 MB/s 2024-09-16T00:32:50.623 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:32:50.682 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-09-16T00:32:50.738 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:32:50.738 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:50.738 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-16T00:32:50.738 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:50.738 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:50.739 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-16 00:32:29.583989909 +0000 2024-09-16T00:32:50.739 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-16 00:28:00.294751989 +0000 2024-09-16T00:32:50.739 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-16 00:28:00.294751989 +0000 2024-09-16T00:32:50.739 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-16 00:28:00.294751989 +0000 2024-09-16T00:32:50.739 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:32:50.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:50.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:50.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:50 smithi102 ceph-mon[35364]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:32:50.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:50.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:32:50.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:50.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:50 smithi102 ceph-mon[35364]: Reconfiguring daemon mon.c on smithi102 2024-09-16T00:32:50.801 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-16T00:32:50.801 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-16T00:32:50.801 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000189753 s, 2.7 MB/s 2024-09-16T00:32:50.803 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:32:50.859 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-09-16T00:32:50.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:50.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:50.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:50 smithi031 ceph-mon[30956]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:32:50.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:32:50.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:32:50.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:50.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:50 smithi031 ceph-mon[30956]: Reconfiguring daemon mon.c on smithi102 2024-09-16T00:32:50.915 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:32:50.916 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:50.916 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-16T00:32:50.916 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:50.916 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:50.916 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-16 00:32:29.583989909 +0000 2024-09-16T00:32:50.916 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-16 00:28:00.655743927 +0000 2024-09-16T00:32:50.916 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-16 00:28:00.655743927 +0000 2024-09-16T00:32:50.916 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-16 00:28:00.655743927 +0000 2024-09-16T00:32:50.916 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000176666 s, 2.9 MB/s 2024-09-16T00:32:50.983 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:32:51.041 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-09-16T00:32:51.096 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:32:51.096 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:32:51.097 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-16T00:32:51.097 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:32:51.097 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:32:51.097 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-16 00:32:29.584989887 +0000 2024-09-16T00:32:51.097 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-16 00:28:00.985736557 +0000 2024-09-16T00:32:51.097 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-16 00:28:00.985736557 +0000 2024-09-16T00:32:51.097 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-16 00:28:00.985736557 +0000 2024-09-16T00:32:51.097 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:32:51.160 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-16T00:32:51.160 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-16T00:32:51.160 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.0001839 s, 2.8 MB/s 2024-09-16T00:32:51.161 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:32:51.218 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-09-16T00:32:51.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:32:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:32:51.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:32:51.604 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:32:51.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:32:51.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:32:51.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:51.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:51.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:51.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:51.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:51.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:51.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:51.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:51.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:32:51.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:32:51.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:32:51.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:52.558 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:52 smithi031 ceph-mon[30956]: Deploying daemon mgr.z on smithi102 2024-09-16T00:32:52.709 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:52 smithi102 ceph-mon[35364]: Deploying daemon mgr.z on smithi102 2024-09-16T00:32:52.710 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:52 smithi102 systemd[1]: Starting Ceph mgr.z for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:32:52.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:52 smithi081 ceph-mon[35539]: Deploying daemon mgr.z on smithi102 2024-09-16T00:32:53.004 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:52 smithi102 podman[37158]: 2024-09-16 00:32:52.709838695 +0000 UTC m=+0.104591471 container create 2f053d2f087db7f9cb336cc9e4a736a6dd429a4f5b79446f62d294c7df6d97fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:53.004 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:52 smithi102 podman[37158]: 2024-09-16 00:32:52.621187951 +0000 UTC m=+0.015940721 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:53.004 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:52 smithi102 podman[37158]: 2024-09-16 00:32:52.903770959 +0000 UTC m=+0.298523732 container init 2f053d2f087db7f9cb336cc9e4a736a6dd429a4f5b79446f62d294c7df6d97fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T00:32:53.004 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:52 smithi102 podman[37158]: 2024-09-16 00:32:52.906597055 +0000 UTC m=+0.301349832 container start 2f053d2f087db7f9cb336cc9e4a736a6dd429a4f5b79446f62d294c7df6d97fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:32:53.004 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:52 smithi102 ceph-mgr[37177]: -- 172.21.15.102:0/758140374 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f2de73d0e0 con 0x55f2de71a800 2024-09-16T00:32:53.004 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 bash[37158]: 2f053d2f087db7f9cb336cc9e4a736a6dd429a4f5b79446f62d294c7df6d97fc 2024-09-16T00:32:53.299 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 systemd[1]: Started Ceph mgr.z for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:32:53.300 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:53.025+0000 7f2453d14040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:32:53.300 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:53.086+0000 7f2453d14040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:32:53.300 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:53.234+0000 7f2453d14040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:32:53.588 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:53.301+0000 7f2453d14040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:32:53.588 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:53.448+0000 7f2453d14040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:32:53.588 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:53.587+0000 7f2453d14040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:32:53.588 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-mon[35364]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:53.588 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:53.589 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:53.589 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:53.589 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:32:53.589 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:32:53.589 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:32:53.589 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:53.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:53 smithi031 ceph-mon[30956]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:32:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:32:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:32:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:53.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:53 smithi081 ceph-mon[35539]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:53.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:53.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:53.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:53.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:32:53.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:32:53.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:32:53.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:32:53.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-09-16T00:32:54.228 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:54.299 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:53.899+0000 7f2453d14040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:32:54.299 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:53.959+0000 7f2453d14040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:32:54.299 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:54 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:54.139+0000 7f2453d14040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:32:54.525 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:54 smithi081 ceph-mon[35539]: Deploying daemon mgr.y on smithi081 2024-09-16T00:32:54.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:54 smithi031 ceph-mon[30956]: Deploying daemon mgr.y on smithi081 2024-09-16T00:32:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.660+0000 7f85a418e640 1 -- 172.21.15.31:0/1630350072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c101c00 msgr2=0x7f859c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.660+0000 7f85a418e640 1 --2- 172.21.15.31:0/1630350072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c101c00 0x7f859c102000 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8598009900 tx=0x7f859802a410 comp rx=0 tx=0).stop 2024-09-16T00:32:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.660+0000 7f85a418e640 1 -- 172.21.15.31:0/1630350072 shutdown_connections 2024-09-16T00:32:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.660+0000 7f85a418e640 1 --2- 172.21.15.31:0/1630350072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f859c1037c0 0x7f859c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.660+0000 7f85a418e640 1 --2- 172.21.15.31:0/1630350072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c102e00 0x7f859c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.660+0000 7f85a418e640 1 --2- 172.21.15.31:0/1630350072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c101c00 0x7f859c102000 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.660+0000 7f85a418e640 1 -- 172.21.15.31:0/1630350072 >> 172.21.15.31:0/1630350072 conn(0x7f859c0fd390 msgr2=0x7f859c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.660+0000 7f85a418e640 1 -- 172.21.15.31:0/1630350072 shutdown_connections 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.660+0000 7f85a418e640 1 -- 172.21.15.31:0/1630350072 wait complete. 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a418e640 1 Processor -- start 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a418e640 1 -- start start 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a418e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f859c101c00 0x7f859c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a418e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c102e00 0x7f859c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a418e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c1037c0 0x7f859c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a418e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10d860 con 0x7f859c1037c0 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a418e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10d6e0 con 0x7f859c101c00 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a418e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10d9e0 con 0x7f859c102e00 2024-09-16T00:32:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a272b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c1037c0 0x7f859c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a272b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c1037c0 0x7f859c19f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45974/0 (socket says 172.21.15.31:45974) 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a272b640 1 -- 172.21.15.31:0/310956765 learned_addr learned my addr 172.21.15.31:0/310956765 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a1f2a640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f859c101c00 0x7f859c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a1729640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c102e00 0x7f859c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a272b640 1 -- 172.21.15.31:0/310956765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c102e00 msgr2=0x7f859c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a272b640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c102e00 0x7f859c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a272b640 1 -- 172.21.15.31:0/310956765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f859c101c00 msgr2=0x7f859c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a272b640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f859c101c00 0x7f859c197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a272b640 1 -- 172.21.15.31:0/310956765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c19fb30 con 0x7f859c1037c0 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a1f2a640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f859c101c00 0x7f859c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.661+0000 7f85a272b640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c1037c0 0x7f859c19f4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f859400eb10 tx=0x7f859400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.662+0000 7f8592ffd640 1 -- 172.21.15.31:0/310956765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594004b20 con 0x7f859c1037c0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.662+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859c19fe20 con 0x7f859c1037c0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.662+0000 7f8592ffd640 1 -- 172.21.15.31:0/310956765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8594004cc0 con 0x7f859c1037c0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.662+0000 7f8592ffd640 1 -- 172.21.15.31:0/310956765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594010760 con 0x7f859c1037c0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.662+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859c1a0280 con 0x7f859c1037c0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.663+0000 7f8592ffd640 1 -- 172.21.15.31:0/310956765 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8594010900 con 0x7f859c1037c0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.663+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859c109000 con 0x7f859c1037c0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.663+0000 7f8592ffd640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8588039500 0x7f858803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.663+0000 7f8592ffd640 1 -- 172.21.15.31:0/310956765 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f859404cd20 con 0x7f859c1037c0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.663+0000 7f85a1f2a640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8588039500 0x7f858803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.663+0000 7f85a1f2a640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8588039500 0x7f858803b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8598005ab0 tx=0x7f8598000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.668+0000 7f8592ffd640 1 -- 172.21.15.31:0/310956765 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8594004fe0 con 0x7f859c1037c0 2024-09-16T00:32:54.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:54 smithi102 ceph-mon[35364]: Deploying daemon mgr.y on smithi081 2024-09-16T00:32:54.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:54.853+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f859c062630 con 0x7f8588039500 2024-09-16T00:32:54.898 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:54 smithi081 systemd[1]: Starting Ceph mgr.y for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:32:55.155 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:54 smithi081 podman[36889]: 2024-09-16 00:32:54.811162872 +0000 UTC m=+0.025901189 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:55.155 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:54 smithi081 podman[36889]: 2024-09-16 00:32:54.899543174 +0000 UTC m=+0.114281492 container create 51d2485aecbbd2a257573d4d9da6df6e17822e111cab671febde51426518e9f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T00:32:55.155 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 podman[36889]: 2024-09-16 00:32:55.127159406 +0000 UTC m=+0.341897723 container init 51d2485aecbbd2a257573d4d9da6df6e17822e111cab671febde51426518e9f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T00:32:55.155 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 podman[36889]: 2024-09-16 00:32:55.130150365 +0000 UTC m=+0.344888696 container start 51d2485aecbbd2a257573d4d9da6df6e17822e111cab671febde51426518e9f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:55.299 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:55.011+0000 7f2453d14040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:32:55.299 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:55.077+0000 7f2453d14040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:32:55.452 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mgr[36908]: -- 172.21.15.81:0/487683906 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf92de90e0 con 0x55cf92dc7400 2024-09-16T00:32:55.452 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 bash[36889]: 51d2485aecbbd2a257573d4d9da6df6e17822e111cab671febde51426518e9f6 2024-09-16T00:32:55.452 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 systemd[1]: Started Ceph mgr.y for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:32:55.452 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:55.251+0000 7f1dd198d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:32:55.452 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:55.311+0000 7f1dd198d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:32:55.747 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:55.467+0000 7f1dd198d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:32:55.747 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:55.536+0000 7f1dd198d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:32:55.747 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:55.698+0000 7f1dd198d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:32:55.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mon[35539]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:55.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:32:55.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:32:55.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:55.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-mon[35364]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:32:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:32:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:55 smithi031 ceph-mon[30956]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:56.103 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:55.836+0000 7f2453d14040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:32:56.104 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:55.952+0000 7f2453d14040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:32:56.104 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.103+0000 7f2453d14040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:32:56.170 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:55.848+0000 7f1dd198d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:32:56.170 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:56.169+0000 7f1dd198d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:32:56.384 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.166+0000 7f2453d14040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:32:56.384 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.316+0000 7f2453d14040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:32:56.384 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.383+0000 7f2453d14040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:32:56.422 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:56.231+0000 7f1dd198d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:32:56.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-mon[35364]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:56.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:56.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:56.684 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.446+0000 7f2453d14040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:32:56.684 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.617+0000 7f2453d14040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:32:56.684 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.683+0000 7f2453d14040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:32:56.747 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:56.422+0000 7f1dd198d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:32:56.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:56 smithi081 ceph-mon[35539]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:56.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:56.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:56.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:56 smithi031 ceph-mon[30956]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:56.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:56.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:56.947 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.749+0000 7f2453d14040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:32:56.947 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.829+0000 7f2453d14040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:32:56.947 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.889+0000 7f2453d14040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:32:56.947 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:56.946+0000 7f2453d14040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:32:57.726 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:57.726 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1920024772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d076c77e-9b90-46b1-8e52-eecb4d02d169"}]: dispatch 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1920024772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d076c77e-9b90-46b1-8e52-eecb4d02d169"}]': finished 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:32:57.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:32:57.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:57.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:57.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:57.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:57.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:57.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:57.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:32:57.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:32:57.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:57.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1920024772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d076c77e-9b90-46b1-8e52-eecb4d02d169"}]: dispatch 2024-09-16T00:32:57.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1920024772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d076c77e-9b90-46b1-8e52-eecb4d02d169"}]': finished 2024-09-16T00:32:57.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:32:57.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:32:57.747 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:57.307+0000 7f1dd198d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:32:57.747 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:57.374+0000 7f1dd198d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:32:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:57.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:57.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:32:57.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:32:57.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:57.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1920024772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d076c77e-9b90-46b1-8e52-eecb4d02d169"}]: dispatch 2024-09-16T00:32:57.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1920024772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d076c77e-9b90-46b1-8e52-eecb4d02d169"}]': finished 2024-09-16T00:32:57.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:32:57.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:32:58.049 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:32:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z[37173]: 2024-09-16T00:32:57.726+0000 7f2453d14040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:32:58.345 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.087+0000 7f1dd198d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:32:58.345 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.202+0000 7f1dd198d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:32:58.345 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.344+0000 7f1dd198d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:32:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:32:58.614+0000 7f8592ffd640 1 -- 172.21.15.31:0/310956765 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7f8594021aa0 con 0x7f859c1037c0 2024-09-16T00:32:58.622 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.405+0000 7f1dd198d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:32:58.622 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.554+0000 7f1dd198d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:32:58.622 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.621+0000 7f1dd198d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:32:58.622 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-mon[35539]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:32:58.622 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-mon[35539]: Reconfiguring daemon mgr.x on smithi031 2024-09-16T00:32:58.622 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:58.622 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-mon[35539]: Standby manager daemon z started 2024-09-16T00:32:58.622 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-mon[35539]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:32:58.623 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-mon[35539]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:32:58.623 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-mon[35539]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:32:58.623 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-mon[35539]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:32:58.623 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/762513236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:32:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:58 smithi102 ceph-mon[35364]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:32:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:58 smithi102 ceph-mon[35364]: Reconfiguring daemon mgr.x on smithi031 2024-09-16T00:32:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:58 smithi102 ceph-mon[35364]: Standby manager daemon z started 2024-09-16T00:32:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:58 smithi102 ceph-mon[35364]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:32:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:58 smithi102 ceph-mon[35364]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:32:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:58 smithi102 ceph-mon[35364]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:32:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:58 smithi102 ceph-mon[35364]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:32:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:58 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/762513236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:32:58.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:58 smithi031 ceph-mon[30956]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:32:58.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:58 smithi031 ceph-mon[30956]: Reconfiguring daemon mgr.x on smithi031 2024-09-16T00:32:58.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:32:58.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:58 smithi031 ceph-mon[30956]: Standby manager daemon z started 2024-09-16T00:32:58.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:58 smithi031 ceph-mon[30956]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:32:58.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:58 smithi031 ceph-mon[30956]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:32:58.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:58 smithi031 ceph-mon[30956]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:32:58.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:58 smithi031 ceph-mon[30956]: from='mgr.? 172.21.15.102:0/3886360583' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:32:58.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:58 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/762513236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:32:58.901 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.681+0000 7f1dd198d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:32:58.901 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.837+0000 7f1dd198d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:32:58.901 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.900+0000 7f1dd198d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:32:59.165 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:58.966+0000 7f1dd198d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:32:59.165 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:59.048+0000 7f1dd198d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:32:59.165 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:59.106+0000 7f1dd198d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:32:59.165 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:59.164+0000 7f1dd198d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:32:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:59 smithi081 ceph-mon[35539]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:59 smithi081 ceph-mon[35539]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-16T00:32:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:32:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:32:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:59 smithi102 ceph-mon[35364]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:59 smithi102 ceph-mon[35364]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-16T00:32:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:32:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:32:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:59 smithi031 ceph-mon[30956]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:32:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:59 smithi031 ceph-mon[30956]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-16T00:32:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:32:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:33:00.247 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:32:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y[36904]: 2024-09-16T00:32:59.945+0000 7f1dd198d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:33:00.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:00.473+0000 7f8592ffd640 1 -- 172.21.15.31:0/310956765 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f859401f340 con 0x7f859c1037c0 2024-09-16T00:33:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:00 smithi081 ceph-mon[35539]: Standby manager daemon y started 2024-09-16T00:33:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:00 smithi081 ceph-mon[35539]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:33:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:00 smithi081 ceph-mon[35539]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:33:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:00 smithi081 ceph-mon[35539]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:33:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:00 smithi081 ceph-mon[35539]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:33:00.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:00 smithi102 ceph-mon[35364]: Standby manager daemon y started 2024-09-16T00:33:00.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:00 smithi102 ceph-mon[35364]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:33:00.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:00 smithi102 ceph-mon[35364]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:33:00.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:00 smithi102 ceph-mon[35364]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:33:00.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:00 smithi102 ceph-mon[35364]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:33:00.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:00 smithi031 ceph-mon[30956]: Standby manager daemon y started 2024-09-16T00:33:00.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:00 smithi031 ceph-mon[30956]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:33:00.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:00 smithi031 ceph-mon[30956]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:33:00.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:00 smithi031 ceph-mon[30956]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:33:00.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:00 smithi031 ceph-mon[30956]: from='mgr.? 172.21.15.81:0/2982490500' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:33:01.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:01 smithi081 ceph-mon[35539]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:01.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:01 smithi081 ceph-mon[35539]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-16T00:33:01.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:33:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:01 smithi102 ceph-mon[35364]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:01 smithi102 ceph-mon[35364]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-16T00:33:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:33:01.863 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:01 smithi031 ceph-mon[30956]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:01.864 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:01 smithi031 ceph-mon[30956]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-16T00:33:01.864 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:33:03.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:03 smithi081 ceph-mon[35539]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:03.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:03 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:03.773 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:03 smithi031 ceph-mon[30956]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:03.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:03 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:03.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:03 smithi102 ceph-mon[35364]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:03.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:03 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:04.640 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:04.640 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:04.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:04.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:04.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:04.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:05.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:05 smithi031 ceph-mon[30956]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:05.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:05 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:05.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:05 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:05.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:05 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:05.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:05 smithi081 ceph-mon[35539]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:05.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:05 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:05.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:05 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:05.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:05 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:05.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:05 smithi102 ceph-mon[35364]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:05.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:05 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:05.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:05 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:05.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:05 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:06.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:06 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:33:06.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:06 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:06.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:06 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:33:06.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:06 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:06.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:06 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:33:06.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:06 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:07.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:07 smithi081 ceph-mon[35539]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:07.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:07 smithi081 ceph-mon[35539]: Deploying daemon osd.0 on smithi031 2024-09-16T00:33:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:07 smithi102 ceph-mon[35364]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:07 smithi102 ceph-mon[35364]: Deploying daemon osd.0 on smithi031 2024-09-16T00:33:07.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:07 smithi031 ceph-mon[30956]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:07.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:07 smithi031 ceph-mon[30956]: Deploying daemon osd.0 on smithi031 2024-09-16T00:33:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:09 smithi081 ceph-mon[35539]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:09.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:09 smithi102 ceph-mon[35364]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:09.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:09 smithi031 ceph-mon[30956]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:11.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:11 smithi081 ceph-mon[35539]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:11.747 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:11 smithi031 ceph-mon[30956]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:11.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:11 smithi102 ceph-mon[35364]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:12.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:12.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:12.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:12.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:12.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:12.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:13 smithi102 ceph-mon[35364]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:13 smithi031 ceph-mon[30956]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:13.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:13 smithi081 ceph-mon[35539]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:15 smithi102 ceph-mon[35364]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:15 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:15 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:15.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:15 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:15.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:15 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:15.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:15 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:15.871 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:15 smithi031 ceph-mon[30956]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:15.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:15 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:15.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:15 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:15.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:15 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:15.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:15 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:15.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:15 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:15.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:15 smithi081 ceph-mon[35539]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:15.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:15 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:15.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:15 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:15.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:15 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:15.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:15 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:15.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:15 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:16.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.181+0000 7f8592ffd640 1 -- 172.21.15.31:0/310956765 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f859c062630 con 0x7f8588039500 2024-09-16T00:33:16.181 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-09-16T00:33:16.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8588039500 msgr2=0x7f858803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:16.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8588039500 0x7f858803b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8598005ab0 tx=0x7f8598000bc0 comp rx=0 tx=0).stop 2024-09-16T00:33:16.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c1037c0 msgr2=0x7f859c19f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c1037c0 0x7f859c19f4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f859400eb10 tx=0x7f859400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:33:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 shutdown_connections 2024-09-16T00:33:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f859c1037c0 0x7f859c19f4c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8588039500 0x7f858803b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c102e00 0x7f859c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 --2- 172.21.15.31:0/310956765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f859c101c00 0x7f859c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 >> 172.21.15.31:0/310956765 conn(0x7f859c0fd390 msgr2=0x7f859c1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 shutdown_connections 2024-09-16T00:33:16.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:33:16.183+0000 7f85a418e640 1 -- 172.21.15.31:0/310956765 wait complete. 2024-09-16T00:33:16.775 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:16.775 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:16.775 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:16.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:16.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:16.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:16.802 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T00:33:16.805 INFO:tasks.cephadm:Deploying osd.1 on smithi081 with /dev/vg_nvme/lv_4... 2024-09-16T00:33:16.805 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:33:16.829 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:16.829 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:16.829 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:17.043 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:33:17.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:17 smithi031 ceph-mon[30956]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:17.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:17 smithi031 ceph-mon[30956]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:33:17.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:33:17 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0[42202]: 2024-09-16T00:33:17.461+0000 7f55351c1640 -1 osd.0 0 log_to_monitors true 2024-09-16T00:33:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:17 smithi081 ceph-mon[35539]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:17 smithi081 ceph-mon[35539]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:33:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:17 smithi102 ceph-mon[35364]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:17 smithi102 ceph-mon[35364]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:33:18.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:18 smithi081 ceph-mon[35539]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:33:18.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:18 smithi081 ceph-mon[35539]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:33:18.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:18 smithi081 ceph-mon[35539]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:33:18.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:18.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:18 smithi081 ceph-mon[35539]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:18.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:18 smithi081 ceph-mon[35539]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:33:18.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:18 smithi031 ceph-mon[30956]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:33:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:18 smithi031 ceph-mon[30956]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:33:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:18 smithi031 ceph-mon[30956]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:33:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:18 smithi031 ceph-mon[30956]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:18 smithi031 ceph-mon[30956]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:33:19.054 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:18 smithi102 ceph-mon[35364]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:33:19.054 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:18 smithi102 ceph-mon[35364]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:33:19.054 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:18 smithi102 ceph-mon[35364]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:33:19.054 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:19.054 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:18 smithi102 ceph-mon[35364]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:19.055 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:18 smithi102 ceph-mon[35364]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:33:19.595 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:33:19 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0[42202]: 2024-09-16T00:33:19.284+0000 7f5531144640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T00:33:19.596 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:33:19 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0[42202]: 2024-09-16T00:33:19.288+0000 7f5528f64640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:33:19.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:19 smithi081 ceph-mon[35539]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:33:19.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:19 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:19.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:19 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:19.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:19 smithi081 ceph-mon[35539]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' 2024-09-16T00:33:19.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:19 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:19.838 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:33:19.866 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_4 2024-09-16T00:33:19.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:19 smithi031 ceph-mon[30956]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:33:19.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:19.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:19.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:19 smithi031 ceph-mon[30956]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' 2024-09-16T00:33:19.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:19 smithi102 ceph-mon[35364]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:33:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:19 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:19 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:19 smithi102 ceph-mon[35364]: from='osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868]' entity='osd.0' 2024-09-16T00:33:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:19 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:20.160 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.613+0000 7f5441939640 1 -- 172.21.15.81:0/3436461608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c101c00 msgr2=0x7f543c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.613+0000 7f5441939640 1 --2- 172.21.15.81:0/3436461608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c101c00 0x7f543c102000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5430009900 tx=0x7f543002a410 comp rx=0 tx=0).stop 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.613+0000 7f5441939640 1 -- 172.21.15.81:0/3436461608 shutdown_connections 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.613+0000 7f5441939640 1 --2- 172.21.15.81:0/3436461608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f543c1037c0 0x7f543c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.613+0000 7f5441939640 1 --2- 172.21.15.81:0/3436461608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f543c102e00 0x7f543c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.613+0000 7f5441939640 1 --2- 172.21.15.81:0/3436461608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c101c00 0x7f543c102000 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.613+0000 7f5441939640 1 -- 172.21.15.81:0/3436461608 >> 172.21.15.81:0/3436461608 conn(0x7f543c0fd390 msgr2=0x7f543c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.613+0000 7f5441939640 1 -- 172.21.15.81:0/3436461608 shutdown_connections 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.613+0000 7f5441939640 1 -- 172.21.15.81:0/3436461608 wait complete. 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f5441939640 1 Processor -- start 2024-09-16T00:33:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f5441939640 1 -- start start 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f5441939640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f543c101c00 0x7f543c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f5441939640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f543c102e00 0x7f543c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f5441939640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c1037c0 0x7f543c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f5441939640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f543c10d8b0 con 0x7f543c101c00 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f5441939640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f543c10d730 con 0x7f543c1037c0 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f5441939640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f543c10da30 con 0x7f543c102e00 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c1037c0 0x7f543c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c1037c0 0x7f543c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50590/0 (socket says 172.21.15.81:50590) 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543b7fe640 1 -- 172.21.15.81:0/455359799 learned_addr learned my addr 172.21.15.81:0/455359799 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-09-16T00:33:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543affd640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f543c101c00 0x7f543c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543a7fc640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f543c102e00 0x7f543c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543b7fe640 1 -- 172.21.15.81:0/455359799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f543c102e00 msgr2=0x7f543c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543b7fe640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f543c102e00 0x7f543c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543b7fe640 1 -- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f543c101c00 msgr2=0x7f543c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543b7fe640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f543c101c00 0x7f543c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.614+0000 7f543b7fe640 1 -- 172.21.15.81:0/455359799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c0753e0 con 0x7f543c1037c0 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.615+0000 7f543b7fe640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c1037c0 0x7f543c074b20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f543400eaa0 tx=0x7f543400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.615+0000 7f5440937640 1 -- 172.21.15.81:0/455359799 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543400cdb0 con 0x7f543c1037c0 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.615+0000 7f5440937640 1 -- 172.21.15.81:0/455359799 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5434005020 con 0x7f543c1037c0 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.615+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543c1a20c0 con 0x7f543c1037c0 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.615+0000 7f5440937640 1 -- 172.21.15.81:0/455359799 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340116b0 con 0x7f543c1037c0 2024-09-16T00:33:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.615+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543c1a2580 con 0x7f543c1037c0 2024-09-16T00:33:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.616+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f543c108fd0 con 0x7f543c1037c0 2024-09-16T00:33:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.617+0000 7f5440937640 1 -- 172.21.15.81:0/455359799 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f5434005560 con 0x7f543c1037c0 2024-09-16T00:33:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.618+0000 7f5440937640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f54240a31b0 0x7f54240a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.618+0000 7f543affd640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f54240a31b0 0x7f54240a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.618+0000 7f5440937640 1 -- 172.21.15.81:0/455359799 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f54340ce030 con 0x7f543c1037c0 2024-09-16T00:33:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.619+0000 7f543affd640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f54240a31b0 0x7f54240a5670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f543002aa00 tx=0x7f54300315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.622+0000 7f5440937640 1 -- 172.21.15.81:0/455359799 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5434016030 con 0x7f543c1037c0 2024-09-16T00:33:20.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:20.797+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f543c062630 con 0x7f54240a31b0 2024-09-16T00:33:21.199 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:33:21.199 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:33:21.199 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi031 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] boot 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:21.200 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:33:21.201 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:33:21.201 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:21.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:33:21.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi031 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] boot 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:33:21.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:21.301 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:33:21.301 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:33:21.301 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:21.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:33:21.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi031 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: osd.0 [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] boot 2024-09-16T00:33:21.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:33:21.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:33:21.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:33:21.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:33:21.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:22.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:22 smithi081 ceph-mon[35539]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:22.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:22 smithi081 ceph-mon[35539]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:33:22.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:22 smithi102 ceph-mon[35364]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:22.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:22 smithi102 ceph-mon[35364]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:33:22.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:22 smithi031 ceph-mon[30956]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:22.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:22 smithi031 ceph-mon[30956]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:33:23.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:23 smithi081 ceph-mon[35539]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:23.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:23 smithi081 ceph-mon[35539]: from='client.? 172.21.15.81:0/805017686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc498a19-f3ac-4145-a121-81e710cd1558"}]: dispatch 2024-09-16T00:33:23.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:23 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc498a19-f3ac-4145-a121-81e710cd1558"}]: dispatch 2024-09-16T00:33:23.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:23 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc498a19-f3ac-4145-a121-81e710cd1558"}]': finished 2024-09-16T00:33:23.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:23 smithi081 ceph-mon[35539]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:33:23.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:23 smithi031 ceph-mon[30956]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:23 smithi031 ceph-mon[30956]: from='client.? 172.21.15.81:0/805017686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc498a19-f3ac-4145-a121-81e710cd1558"}]: dispatch 2024-09-16T00:33:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:23 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc498a19-f3ac-4145-a121-81e710cd1558"}]: dispatch 2024-09-16T00:33:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:23 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc498a19-f3ac-4145-a121-81e710cd1558"}]': finished 2024-09-16T00:33:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:23 smithi031 ceph-mon[30956]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:33:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:23.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:23 smithi102 ceph-mon[35364]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:23.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:23 smithi102 ceph-mon[35364]: from='client.? 172.21.15.81:0/805017686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc498a19-f3ac-4145-a121-81e710cd1558"}]: dispatch 2024-09-16T00:33:23.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:23 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc498a19-f3ac-4145-a121-81e710cd1558"}]: dispatch 2024-09-16T00:33:23.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:23 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc498a19-f3ac-4145-a121-81e710cd1558"}]': finished 2024-09-16T00:33:23.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:23 smithi102 ceph-mon[35364]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:33:23.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:24.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:24 smithi031 ceph-mon[30956]: from='client.? 172.21.15.81:0/4184334896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:33:24.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:24 smithi081 ceph-mon[35539]: from='client.? 172.21.15.81:0/4184334896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:33:24.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:24 smithi102 ceph-mon[35364]: from='client.? 172.21.15.81:0/4184334896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:33:25.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:25 smithi031 ceph-mon[30956]: pgmap v43: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:25.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:25 smithi081 ceph-mon[35539]: pgmap v43: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:25.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:25 smithi102 ceph-mon[35364]: pgmap v43: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:27.628 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:27 smithi031 ceph-mon[30956]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:27.644 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:27 smithi081 ceph-mon[35539]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:27.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:27 smithi102 ceph-mon[35364]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:29.495 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:29 smithi081 ceph-mon[35539]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:29.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:29 smithi031 ceph-mon[30956]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:29 smithi102 ceph-mon[35364]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:30.615 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:30 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:33:30.615 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:30 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:30.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:30 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:33:30.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:30 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:30.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:30 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:33:30.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:30 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:31.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:31 smithi031 ceph-mon[30956]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:31.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:31 smithi031 ceph-mon[30956]: Deploying daemon osd.1 on smithi081 2024-09-16T00:33:31.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:31 smithi081 ceph-mon[35539]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:31.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:31 smithi081 ceph-mon[35539]: Deploying daemon osd.1 on smithi081 2024-09-16T00:33:31.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:31 smithi102 ceph-mon[35364]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:31.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:31 smithi102 ceph-mon[35364]: Deploying daemon osd.1 on smithi081 2024-09-16T00:33:33.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:33 smithi031 ceph-mon[30956]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:33.642 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:33 smithi081 ceph-mon[35539]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:33.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:33 smithi102 ceph-mon[35364]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:35.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:35 smithi031 ceph-mon[30956]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:35.696 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:35 smithi081 ceph-mon[35539]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:35 smithi102 ceph-mon[35364]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:37.595 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:37 smithi081 ceph-mon[35539]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:37.595 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:37 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:37.596 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:37 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:37.596 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:37 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:37.601 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:37 smithi102 ceph-mon[35364]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:37.601 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:37 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:37.601 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:37 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:37.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:37 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:37.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:37 smithi031 ceph-mon[30956]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:37.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:37 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:37.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:37 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:37.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:37 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:39.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:39 smithi031 ceph-mon[30956]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:39.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:39 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:39.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:39 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:39.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:39 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:39.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:39 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:39.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:39 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:39.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:39 smithi081 ceph-mon[35539]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:39.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:39.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:39.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:39.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:39.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:39.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:39 smithi102 ceph-mon[35364]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:39.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:39 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:39.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:39 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:39.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:39 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:39.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:39 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:39.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:39 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:40.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.763+0000 7f5440937640 1 -- 172.21.15.81:0/455359799 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f543c062630 con 0x7f54240a31b0 2024-09-16T00:33:40.764 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 1 on host 'smithi081' 2024-09-16T00:33:40.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f54240a31b0 msgr2=0x7f54240a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:40.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f54240a31b0 0x7f54240a5670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f543002aa00 tx=0x7f54300315b0 comp rx=0 tx=0).stop 2024-09-16T00:33:40.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c1037c0 msgr2=0x7f543c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:40.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c1037c0 0x7f543c074b20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f543400eaa0 tx=0x7f543400ef60 comp rx=0 tx=0).stop 2024-09-16T00:33:40.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 shutdown_connections 2024-09-16T00:33:40.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f543c1037c0 0x7f543c074b20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:40.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f54240a31b0 0x7f54240a5670 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:40.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f543c102e00 0x7f543c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:40.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 --2- 172.21.15.81:0/455359799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f543c101c00 0x7f543c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:40.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 >> 172.21.15.81:0/455359799 conn(0x7f543c0fd390 msgr2=0x7f543c1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:40.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 shutdown_connections 2024-09-16T00:33:40.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:33:40.766+0000 7f5441939640 1 -- 172.21.15.81:0/455359799 wait complete. 2024-09-16T00:33:41.352 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:41 smithi081 ceph-mon[35539]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:41.353 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:41.353 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:41.353 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:41.378 DEBUG:teuthology.orchestra.run.smithi081:osd.1> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T00:33:41.380 INFO:tasks.cephadm:Deploying osd.2 on smithi102 with /dev/vg_nvme/lv_4... 2024-09-16T00:33:41.381 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:33:41.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:41 smithi031 ceph-mon[30956]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:41.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:41.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:41.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:41.628 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:33:41.749 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:41 smithi102 ceph-mon[35364]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:41.749 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:41.749 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:41.749 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:42.783 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:33:42 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1[40003]: 2024-09-16T00:33:42.629+0000 7f4000a1a640 -1 osd.1 0 log_to_monitors true 2024-09-16T00:33:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:43 smithi102 ceph-mon[35364]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:43 smithi102 ceph-mon[35364]: from='osd.1 [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:33:43.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:43 smithi102 ceph-mon[35364]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:33:43.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:43 smithi031 ceph-mon[30956]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:43.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:43 smithi031 ceph-mon[30956]: from='osd.1 [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:33:43.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:43 smithi031 ceph-mon[30956]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:33:43.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:43 smithi081 ceph-mon[35539]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:43.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:43 smithi081 ceph-mon[35539]: from='osd.1 [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:33:43.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:43 smithi081 ceph-mon[35539]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:33:43.822 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:33:43.846 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2024-09-16T00:33:44.088 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:33:44.464 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:33:44.464 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='osd.1 [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:33:44.464 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:33:44.464 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:44.464 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:33:44.464 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:33:44.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:44.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:44.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:33:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='osd.1 [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:33:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:33:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:33:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:33:44.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:44.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:44.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 -- 172.21.15.102:0/3813337080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf10102eb0 msgr2=0x7faf10103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 --2- 172.21.15.102:0/3813337080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf10102eb0 0x7faf10103330 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faf04009930 tx=0x7faf0402a420 comp rx=0 tx=0).stop 2024-09-16T00:33:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 -- 172.21.15.102:0/3813337080 shutdown_connections 2024-09-16T00:33:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 --2- 172.21.15.102:0/3813337080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf10103870 0x7faf1010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 --2- 172.21.15.102:0/3813337080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf10102eb0 0x7faf10103330 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 --2- 172.21.15.102:0/3813337080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf100ffed0 0x7faf101002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 -- 172.21.15.102:0/3813337080 >> 172.21.15.102:0/3813337080 conn(0x7faf100fba40 msgr2=0x7faf100fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 -- 172.21.15.102:0/3813337080 shutdown_connections 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 -- 172.21.15.102:0/3813337080 wait complete. 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 Processor -- start 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.546+0000 7faf1609d640 1 -- start start 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf1609d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf100ffed0 0x7faf10197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf1609d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf10102eb0 0x7faf10198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf1609d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf10103870 0x7faf1019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf1609d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010d8b0 con 0x7faf10102eb0 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf1609d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010d730 con 0x7faf10103870 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf1609d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010da30 con 0x7faf100ffed0 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf100ffed0 0x7faf10197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf100ffed0 0x7faf10197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54362/0 (socket says 172.21.15.102:54362) 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0f7fe640 1 -- 172.21.15.102:0/3661140144 learned_addr learned my addr 172.21.15.102:0/3661140144 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0ffff640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf10103870 0x7faf1019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0effd640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf10102eb0 0x7faf10198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0f7fe640 1 -- 172.21.15.102:0/3661140144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf10103870 msgr2=0x7faf1019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0f7fe640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf10103870 0x7faf1019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0f7fe640 1 -- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf10102eb0 msgr2=0x7faf10198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0f7fe640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf10102eb0 0x7faf10198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0f7fe640 1 -- 172.21.15.102:0/3661140144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1019fb20 con 0x7faf100ffed0 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0f7fe640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf100ffed0 0x7faf10197ef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf0000ccb0 tx=0x7faf0000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0cff9640 1 -- 172.21.15.102:0/3661140144 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00003b70 con 0x7faf100ffed0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0cff9640 1 -- 172.21.15.102:0/3661140144 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0000ae50 con 0x7faf100ffed0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1019fe10 con 0x7faf100ffed0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf0cff9640 1 -- 172.21.15.102:0/3661140144 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0000f810 con 0x7faf100ffed0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.547+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf101a0270 con 0x7faf100ffed0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.548+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf10109050 con 0x7faf100ffed0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.550+0000 7faf0cff9640 1 -- 172.21.15.102:0/3661140144 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7faf0000f9b0 con 0x7faf100ffed0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.550+0000 7faf0cff9640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faef80a3260 0x7faef80a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.550+0000 7faf0cff9640 1 -- 172.21.15.102:0/3661140144 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7faf000ce320 con 0x7faf100ffed0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.550+0000 7faf0effd640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faef80a3260 0x7faef80a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.551+0000 7faf0effd640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faef80a3260 0x7faef80a5720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf04006010 tx=0x7faf040315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.553+0000 7faf0cff9640 1 -- 172.21.15.102:0/3661140144 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf00014070 con 0x7faf100ffed0 2024-09-16T00:33:44.638 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:33:44.638 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='osd.1 [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:33:44.638 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:33:44.638 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:44.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:33:44.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:33:44.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:44.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:44.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:33:44.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:33:44.732+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7faf10062630 con 0x7faef80a3260 2024-09-16T00:33:45.352 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:45.353 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1[40003]: 2024-09-16T00:33:45.029+0000 7f3ffc99d640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T00:33:45.353 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1[40003]: 2024-09-16T00:33:45.036+0000 7f3ff47bd640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:33:45.441 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi081 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi081 to 17088M 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: from='osd.1 ' entity='osd.1' 2024-09-16T00:33:45.442 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:45.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi081 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi081 to 17088M 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: from='osd.1 ' entity='osd.1' 2024-09-16T00:33:45.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi081 2024-09-16T00:33:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi081 to 17088M 2024-09-16T00:33:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:33:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:33:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:33:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:33:45.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:45.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: from='osd.1 ' entity='osd.1' 2024-09-16T00:33:45.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:46.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:46 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:33:46.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:46 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:33:46.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:46 smithi102 ceph-mon[35364]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:46.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:46 smithi102 ceph-mon[35364]: osd.1 [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] boot 2024-09-16T00:33:46.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:46 smithi102 ceph-mon[35364]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:33:46.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:46.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:46 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:33:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:46 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:33:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:46 smithi031 ceph-mon[30956]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:46 smithi031 ceph-mon[30956]: osd.1 [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] boot 2024-09-16T00:33:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:46 smithi031 ceph-mon[30956]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:33:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:46.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:46 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:33:46.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:46 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:33:46.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:46 smithi081 ceph-mon[35539]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:46.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:46 smithi081 ceph-mon[35539]: osd.1 [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] boot 2024-09-16T00:33:46.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:46 smithi081 ceph-mon[35539]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:33:46.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:33:47.510 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:47 smithi102 ceph-mon[35364]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:47.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:47 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/2047322214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761075fe-1766-4924-afa3-6e2b1b3c3a37"}]: dispatch 2024-09-16T00:33:47.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:47 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761075fe-1766-4924-afa3-6e2b1b3c3a37"}]: dispatch 2024-09-16T00:33:47.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:47 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "761075fe-1766-4924-afa3-6e2b1b3c3a37"}]': finished 2024-09-16T00:33:47.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:47 smithi102 ceph-mon[35364]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T00:33:47.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:33:47.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:47 smithi031 ceph-mon[30956]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:47.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:47 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/2047322214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761075fe-1766-4924-afa3-6e2b1b3c3a37"}]: dispatch 2024-09-16T00:33:47.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:47 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761075fe-1766-4924-afa3-6e2b1b3c3a37"}]: dispatch 2024-09-16T00:33:47.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:47 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "761075fe-1766-4924-afa3-6e2b1b3c3a37"}]': finished 2024-09-16T00:33:47.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:47 smithi031 ceph-mon[30956]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T00:33:47.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:33:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:47 smithi081 ceph-mon[35539]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:33:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:47 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/2047322214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761075fe-1766-4924-afa3-6e2b1b3c3a37"}]: dispatch 2024-09-16T00:33:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:47 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761075fe-1766-4924-afa3-6e2b1b3c3a37"}]: dispatch 2024-09-16T00:33:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:47 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "761075fe-1766-4924-afa3-6e2b1b3c3a37"}]': finished 2024-09-16T00:33:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:47 smithi081 ceph-mon[35539]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T00:33:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:33:48.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:48 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/3299360293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:33:48.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:48 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/3299360293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:33:48.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:48 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/3299360293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:33:49.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:49 smithi031 ceph-mon[30956]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:49.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:49 smithi081 ceph-mon[35539]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:49.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:49 smithi102 ceph-mon[35364]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:51.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:51 smithi031 ceph-mon[30956]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:51.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:51 smithi102 ceph-mon[35364]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:51 smithi081 ceph-mon[35539]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:53.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:53 smithi102 ceph-mon[35364]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:53 smithi031 ceph-mon[30956]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:53.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:53 smithi081 ceph-mon[35539]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:55.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:55 smithi031 ceph-mon[30956]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:55.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:33:55.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:55.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:55 smithi081 ceph-mon[35539]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:55.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:33:55.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:55.766 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:55 smithi102 ceph-mon[35364]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:55.766 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:33:55.766 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:56.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:56 smithi102 ceph-mon[35364]: Deploying daemon osd.2 on smithi102 2024-09-16T00:33:56.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:56 smithi081 ceph-mon[35539]: Deploying daemon osd.2 on smithi102 2024-09-16T00:33:56.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:56 smithi031 ceph-mon[30956]: Deploying daemon osd.2 on smithi102 2024-09-16T00:33:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:57 smithi102 ceph-mon[35364]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:57 smithi081 ceph-mon[35539]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:57.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:57 smithi031 ceph-mon[30956]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:33:59 smithi081 ceph-mon[35539]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:33:59 smithi102 ceph-mon[35364]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:33:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:33:59 smithi031 ceph-mon[30956]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:01.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:01 smithi102 ceph-mon[35364]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:01.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:01 smithi081 ceph-mon[35539]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:01.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:01 smithi031 ceph-mon[30956]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:02.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:02.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:02.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:02.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:02.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:02.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:02.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:03.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:03 smithi081 ceph-mon[35539]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:03.785 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:03 smithi102 ceph-mon[35364]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:03.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:03 smithi031 ceph-mon[30956]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:04.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:04.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:04.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:04.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:04.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:04.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:04.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:04.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:04.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:04.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:04.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:04.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:04.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:04.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:04.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:05.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:05 smithi102 ceph-mon[35364]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:05.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:05 smithi081 ceph-mon[35539]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:05.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:05 smithi031 ceph-mon[30956]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:05.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.957+0000 7faf0cff9640 1 -- 172.21.15.102:0/3661140144 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faf10062630 con 0x7faef80a3260 2024-09-16T00:34:05.959 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 2 on host 'smithi102' 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faef80a3260 msgr2=0x7faef80a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faef80a3260 0x7faef80a5720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf04006010 tx=0x7faf040315b0 comp rx=0 tx=0).stop 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf100ffed0 msgr2=0x7faf10197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf100ffed0 0x7faf10197ef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf0000ccb0 tx=0x7faf0000a620 comp rx=0 tx=0).stop 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 shutdown_connections 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf10103870 0x7faf1019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf10102eb0 0x7faf10198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faef80a3260 0x7faef80a5720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 --2- 172.21.15.102:0/3661140144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf100ffed0 0x7faf10197ef0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 >> 172.21.15.102:0/3661140144 conn(0x7faf100fba40 msgr2=0x7faf10106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:05.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.959+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 shutdown_connections 2024-09-16T00:34:05.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:34:05.960+0000 7faf1609d640 1 -- 172.21.15.102:0/3661140144 wait complete. 2024-09-16T00:34:06.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:06 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:06.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:06 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:06.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:06 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:06.672 DEBUG:teuthology.orchestra.run.smithi102:osd.2> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T00:34:06.674 INFO:tasks.cephadm:Deploying osd.3 on smithi031 with /dev/vg_nvme/lv_3... 2024-09-16T00:34:06.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:34:06.700 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:06 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:06.700 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:06 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:06.700 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:06 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:06.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:06 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:06.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:06 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:06.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:06 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:06.926 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:07.300 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:07 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2[40340]: 2024-09-16T00:34:07.106+0000 7fe6d5db0640 -1 osd.2 0 log_to_monitors true 2024-09-16T00:34:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:07 smithi031 ceph-mon[30956]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:07.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:07 smithi031 ceph-mon[30956]: from='osd.2 [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:34:07.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:07 smithi031 ceph-mon[30956]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:34:07.685 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:07 smithi102 ceph-mon[35364]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:07.685 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:07 smithi102 ceph-mon[35364]: from='osd.2 [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:34:07.685 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:07 smithi102 ceph-mon[35364]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:34:07.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:07 smithi081 ceph-mon[35539]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:07.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:07 smithi081 ceph-mon[35539]: from='osd.2 [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:34:07.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:07 smithi081 ceph-mon[35539]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:34:08.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:08 smithi102 ceph-mon[35364]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:34:08.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:08 smithi102 ceph-mon[35364]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:34:08.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:08.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:08 smithi102 ceph-mon[35364]: from='osd.2 [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:34:08.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:08 smithi102 ceph-mon[35364]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:34:08.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:08 smithi031 ceph-mon[30956]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:34:08.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:08 smithi031 ceph-mon[30956]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:34:08.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:08.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:08 smithi031 ceph-mon[30956]: from='osd.2 [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:34:08.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:08 smithi031 ceph-mon[30956]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:34:08.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:08 smithi081 ceph-mon[35539]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:34:08.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:08 smithi081 ceph-mon[35539]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:34:08.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:08.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:08 smithi081 ceph-mon[35539]: from='osd.2 [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:34:08.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:08 smithi081 ceph-mon[35539]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:34:09.299 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2[40340]: 2024-09-16T00:34:09.126+0000 7fe6d1d33640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T00:34:09.299 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2[40340]: 2024-09-16T00:34:09.133+0000 7fe6c9b53640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:34:09.656 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:09.656 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:34:09.656 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='osd.2 ' entity='osd.2' 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:09.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:09.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:34:09.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:34:09.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:09.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:09.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='osd.2 ' entity='osd.2' 2024-09-16T00:34:09.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:34:09.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:09.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:09.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.726 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:34:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:34:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:34:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='osd.2 ' entity='osd.2' 2024-09-16T00:34:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:34:09.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:09.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:09.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:09.752 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-09-16T00:34:10.046 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.630+0000 7f072ad1e640 1 -- 172.21.15.31:0/2319838155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 msgr2=0x7f072410c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.630+0000 7f072ad1e640 1 --2- 172.21.15.31:0/2319838155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 0x7f072410c240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f071c009ca0 tx=0x7f071c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.630+0000 7f072ad1e640 1 -- 172.21.15.31:0/2319838155 shutdown_connections 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.630+0000 7f072ad1e640 1 --2- 172.21.15.31:0/2319838155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 0x7f072410c240 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.630+0000 7f072ad1e640 1 --2- 172.21.15.31:0/2319838155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724104f60 0x7f07241053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.630+0000 7f072ad1e640 1 --2- 172.21.15.31:0/2319838155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724103e00 0x7f0724104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.630+0000 7f072ad1e640 1 -- 172.21.15.31:0/2319838155 >> 172.21.15.31:0/2319838155 conn(0x7f07240ff5b0 msgr2=0x7f07241019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.630+0000 7f072ad1e640 1 -- 172.21.15.31:0/2319838155 shutdown_connections 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.630+0000 7f072ad1e640 1 -- 172.21.15.31:0/2319838155 wait complete. 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f072ad1e640 1 Processor -- start 2024-09-16T00:34:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f072ad1e640 1 -- start start 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f072ad1e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724103e00 0x7f072419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f072ad1e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724104f60 0x7f072419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f072ad1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 0x7f07241a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f072ad1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f072410fb30 con 0x7f0724105920 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f072ad1e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f072410f9b0 con 0x7f0724104f60 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f072ad1e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f072410fcb0 con 0x7f0724103e00 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f07292bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 0x7f07241a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f07292bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 0x7f07241a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57078/0 (socket says 172.21.15.31:57078) 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f07292bb640 1 -- 172.21.15.31:0/4280049192 learned_addr learned my addr 172.21.15.31:0/4280049192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:34:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f07292bb640 1 -- 172.21.15.31:0/4280049192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724103e00 msgr2=0x7f072419a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f0728aba640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724103e00 0x7f072419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f07292bb640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724103e00 0x7f072419a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f07292bb640 1 -- 172.21.15.31:0/4280049192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724104f60 msgr2=0x7f072419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f07292bb640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724104f60 0x7f072419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f07292bb640 1 -- 172.21.15.31:0/4280049192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07241a1c90 con 0x7f0724105920 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.631+0000 7f0728aba640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724103e00 0x7f072419a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.632+0000 7f07292bb640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 0x7f07241a1620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f071c0098d0 tx=0x7f071c02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.632+0000 7f0721ffb640 1 -- 172.21.15.31:0/4280049192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c02fc30 con 0x7f0724105920 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.632+0000 7f0721ffb640 1 -- 172.21.15.31:0/4280049192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c003e50 con 0x7f0724105920 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.632+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07241a1f20 con 0x7f0724105920 2024-09-16T00:34:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.632+0000 7f0721ffb640 1 -- 172.21.15.31:0/4280049192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c0305c0 con 0x7f0724105920 2024-09-16T00:34:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.632+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07241a24b0 con 0x7f0724105920 2024-09-16T00:34:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.633+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f072410b120 con 0x7f0724105920 2024-09-16T00:34:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.634+0000 7f0721ffb640 1 -- 172.21.15.31:0/4280049192 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f071c004b40 con 0x7f0724105920 2024-09-16T00:34:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.635+0000 7f0721ffb640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07140a31e0 0x7f07140a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.635+0000 7f0721ffb640 1 -- 172.21.15.31:0/4280049192 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2851+0+0 (secure 0 0 0) 0x7f071c0ef830 con 0x7f0724105920 2024-09-16T00:34:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.635+0000 7f0728aba640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07140a31e0 0x7f07140a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.635+0000 7f0728aba640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07140a31e0 0x7f07140a56a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f071800baa0 tx=0x7f0718005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.638+0000 7f0721ffb640 1 -- 172.21.15.31:0/4280049192 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f071c0bdb80 con 0x7f0724105920 2024-09-16T00:34:10.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:10 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:34:10.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:10 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:34:10.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:10 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi102 2024-09-16T00:34:10.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:10 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi102 to 17088M 2024-09-16T00:34:10.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:10 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:10.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:10 smithi081 ceph-mon[35539]: osd.2 [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] boot 2024-09-16T00:34:10.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:10 smithi081 ceph-mon[35539]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T00:34:10.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:10 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:10.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:10 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:34:10.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:10 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:34:10.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:10 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi102 2024-09-16T00:34:10.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:10 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi102 to 17088M 2024-09-16T00:34:10.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:10 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:10.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:10 smithi102 ceph-mon[35364]: osd.2 [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] boot 2024-09-16T00:34:10.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:10 smithi102 ceph-mon[35364]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T00:34:10.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:10 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:10.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:10.827+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0724062630 con 0x7f07140a31e0 2024-09-16T00:34:10.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:10 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:34:10.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:10 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:34:10.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:10 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi102 2024-09-16T00:34:10.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:10 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi102 to 17088M 2024-09-16T00:34:10.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:10 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:10.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:10 smithi031 ceph-mon[30956]: osd.2 [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] boot 2024-09-16T00:34:10.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:10 smithi031 ceph-mon[30956]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T00:34:10.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:10 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:34:11.571 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:11 smithi031 ceph-mon[30956]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:11.571 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:34:11.571 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:34:11.571 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:11.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:11 smithi081 ceph-mon[35539]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:11.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:34:11.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:34:11.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:11.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:11 smithi102 ceph-mon[35364]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:34:11.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:34:11.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:34:11.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:12.610 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:12 smithi031 ceph-mon[30956]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:12.610 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:12 smithi031 ceph-mon[30956]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:34:12.610 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:34:12.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:12 smithi081 ceph-mon[35539]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:12.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:12 smithi081 ceph-mon[35539]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:34:12.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:34:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:12 smithi102 ceph-mon[35364]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:12 smithi102 ceph-mon[35364]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:34:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:34:13.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:13 smithi081 ceph-mon[35539]: pgmap v75: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:13.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:13 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:34:13.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:13 smithi081 ceph-mon[35539]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:34:13.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:13 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:34:13.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:13 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2132556129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f362961-16d5-4892-99df-3bec7a940e68"}]: dispatch 2024-09-16T00:34:13.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:13 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:34:13.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:13 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2132556129' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f362961-16d5-4892-99df-3bec7a940e68"}]': finished 2024-09-16T00:34:13.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:13 smithi081 ceph-mon[35539]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T00:34:13.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:13 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:13 smithi102 ceph-mon[35364]: pgmap v75: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:13.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:13 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:34:13.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:13 smithi102 ceph-mon[35364]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:34:13.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:13 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:34:13.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:13 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2132556129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f362961-16d5-4892-99df-3bec7a940e68"}]: dispatch 2024-09-16T00:34:13.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:13 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:34:13.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:13 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2132556129' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f362961-16d5-4892-99df-3bec7a940e68"}]': finished 2024-09-16T00:34:13.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:13 smithi102 ceph-mon[35364]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T00:34:13.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:13 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:13.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:13 smithi031 ceph-mon[30956]: pgmap v75: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:13 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:34:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:13 smithi031 ceph-mon[30956]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:34:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:13 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:34:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:13 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2132556129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f362961-16d5-4892-99df-3bec7a940e68"}]: dispatch 2024-09-16T00:34:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:13 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:34:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:13 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2132556129' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f362961-16d5-4892-99df-3bec7a940e68"}]': finished 2024-09-16T00:34:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:13 smithi031 ceph-mon[30956]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T00:34:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:13 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:14.342 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:34:13 smithi031 sudo[47078]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:34:14.342 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:34:13 smithi031 sudo[47078]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:14.342 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:34:13 smithi031 sudo[47078]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:14.342 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:34:13 smithi031 sudo[47078]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:14.466+0000 7f0721ffb640 1 -- 172.21.15.31:0/4280049192 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f071c004000 con 0x7f0724105920 2024-09-16T00:34:14.625 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:34:14 smithi031 sudo[47127]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:34:14.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:34:14 smithi031 sudo[47127]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:14.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:34:14 smithi031 sudo[47127]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:14.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:34:14 smithi031 sudo[47127]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:14.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:14 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2356511623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:34:14.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:14 smithi031 ceph-mon[30956]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:34:14.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:14 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:14.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:14 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:14.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:14 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2356511623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:34:14.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:14 smithi081 ceph-mon[35539]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:34:14.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:14 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:14.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:14 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:14.748 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:34:14 smithi081 sudo[43845]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:34:14.748 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:34:14 smithi081 sudo[43845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:14.748 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:34:14 smithi081 sudo[43845]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:14.748 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:34:14 smithi081 sudo[43845]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:14.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:14 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2356511623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:34:14.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:14 smithi102 ceph-mon[35364]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:34:14.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:14 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:14.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:14 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:15.247 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:34:14 smithi081 sudo[43848]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:34:15.247 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:34:14 smithi081 sudo[43848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:15.247 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:34:14 smithi081 sudo[43848]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:15.247 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:34:14 smithi081 sudo[43848]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:15.392 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:15 smithi102 sudo[43982]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:34:15.392 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:15 smithi102 sudo[43982]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:15.392 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:15 smithi102 sudo[43982]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:15.392 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:15 smithi102 sudo[43982]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:15.799 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:15 smithi102 sudo[43985]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:34:15.800 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:15 smithi102 sudo[43985]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:15.800 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:15 smithi102 sudo[43985]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:15.800 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:34:15 smithi102 sudo[43985]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:16.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:15 smithi031 sudo[47506]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:34:16.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:15 smithi031 sudo[47506]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:16.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:15 smithi031 sudo[47506]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:16.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:15 smithi031 sudo[47506]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:16.588 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 ceph-mon[35364]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:16.588 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 ceph-mon[35364]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:34:16.588 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:16.588 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:16.588 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 ceph-mon[35364]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:34:16.588 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 sudo[43988]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:34:16.589 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 sudo[43988]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:16.589 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 sudo[43988]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:16.589 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 sudo[43988]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:16.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:16 smithi031 sudo[47693]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:34:16.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:16 smithi031 sudo[47693]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:16.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:16 smithi031 sudo[47693]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:16.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:16 smithi031 sudo[47693]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:16.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:16 smithi031 ceph-mon[30956]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:16.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:16 smithi031 ceph-mon[30956]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:34:16.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:16.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:16.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:16 smithi031 ceph-mon[30956]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:34:16.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:16 smithi081 ceph-mon[35539]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:16.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:16 smithi081 ceph-mon[35539]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:34:16.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:16.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:16.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:16 smithi081 ceph-mon[35539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:34:17.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 sudo[43991]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:34:17.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 sudo[43991]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:17.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 sudo[43991]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:17.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:16 smithi102 sudo[43991]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:17.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 sudo[43851]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:34:17.324 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 sudo[43851]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:17.324 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 sudo[43851]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:17.324 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 sudo[43851]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:34:17.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:34:17.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:34:17.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:34:17.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:34:17.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:34:17.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:17 smithi031 ceph-mon[30956]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:34:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 sudo[43854]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:34:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 sudo[43854]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:34:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 sudo[43854]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:34:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 sudo[43854]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:34:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:34:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:34:17.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:34:17.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:34:17.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:17 smithi081 ceph-mon[35539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:34:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:34:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:34:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:34:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:17 smithi102 ceph-mon[35364]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:34:19.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:19 smithi031 ceph-mon[30956]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:19.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:19 smithi081 ceph-mon[35539]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:19 smithi102 ceph-mon[35364]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:21.604 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:21 smithi031 ceph-mon[30956]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:21.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:21 smithi081 ceph-mon[35539]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:21.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:21 smithi102 ceph-mon[35364]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:22.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:34:22.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:22.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:22 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:34:22.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:22 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:22.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:22 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:34:22.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:22 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:23.588 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:23 smithi031 ceph-mon[30956]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-16T00:34:23.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:23 smithi031 ceph-mon[30956]: Deploying daemon osd.3 on smithi031 2024-09-16T00:34:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:23 smithi081 ceph-mon[35539]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-16T00:34:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:23 smithi081 ceph-mon[35539]: Deploying daemon osd.3 on smithi031 2024-09-16T00:34:23.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:23 smithi102 ceph-mon[35364]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-16T00:34:23.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:23 smithi102 ceph-mon[35364]: Deploying daemon osd.3 on smithi031 2024-09-16T00:34:25.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:25 smithi031 ceph-mon[30956]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T00:34:25.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:25 smithi081 ceph-mon[35539]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T00:34:25.803 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:25 smithi102 ceph-mon[35364]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T00:34:27.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:27 smithi031 ceph-mon[30956]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T00:34:27.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:27 smithi081 ceph-mon[35539]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T00:34:27.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:27 smithi102 ceph-mon[35364]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T00:34:28.614 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:28 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:28.615 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:28 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:28.615 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:28 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:28.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:28 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:28.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:28 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:28.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:28 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:28.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:28 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:28.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:28 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:28.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:28 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:29.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:29 smithi031 ceph-mon[30956]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T00:34:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:29 smithi081 ceph-mon[35539]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T00:34:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:29 smithi102 ceph-mon[35364]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T00:34:31.728 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:31 smithi031 ceph-mon[30956]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:34:31.729 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:31.729 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:31.729 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:31.729 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:31.729 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:31.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.768+0000 7f0721ffb640 1 -- 172.21.15.31:0/4280049192 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0724062630 con 0x7f07140a31e0 2024-09-16T00:34:31.769 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 3 on host 'smithi031' 2024-09-16T00:34:31.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07140a31e0 msgr2=0x7f07140a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:31.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07140a31e0 0x7f07140a56a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f071800baa0 tx=0x7f0718005d90 comp rx=0 tx=0).stop 2024-09-16T00:34:31.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 msgr2=0x7f07241a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:31.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 0x7f07241a1620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f071c0098d0 tx=0x7f071c02f840 comp rx=0 tx=0).stop 2024-09-16T00:34:31.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 shutdown_connections 2024-09-16T00:34:31.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724105920 0x7f07241a1620 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:31.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724104f60 0x7f072419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:31.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07140a31e0 0x7f07140a56a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:31.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 --2- 172.21.15.31:0/4280049192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724103e00 0x7f072419a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:31.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 >> 172.21.15.31:0/4280049192 conn(0x7f07240ff5b0 msgr2=0x7f07241082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:31.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 shutdown_connections 2024-09-16T00:34:31.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:34:31.771+0000 7f072ad1e640 1 -- 172.21.15.31:0/4280049192 wait complete. 2024-09-16T00:34:31.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:31 smithi102 ceph-mon[35364]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:34:31.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:31.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:31.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:31.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:31.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:31.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:31 smithi081 ceph-mon[35539]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:34:32.000 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:32.001 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:32.001 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:32.001 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:32.001 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:32.395 DEBUG:teuthology.orchestra.run.smithi031:osd.3> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T00:34:32.398 INFO:tasks.cephadm:Deploying osd.4 on smithi081 with /dev/vg_nvme/lv_3... 2024-09-16T00:34:32.398 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:34:32.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:32 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:32.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:32 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:32.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:32 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:32.652 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:34:32.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:32 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:32.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:32 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:32.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:32 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:32.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:32 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:32.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:32 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:32.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:32 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:33.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:33 smithi102 ceph-mon[35364]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:34:33.865 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:33 smithi031 ceph-mon[30956]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:34:33.937 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:33 smithi081 ceph-mon[35539]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:34:34.118 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:34:33 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3[49114]: 2024-09-16T00:34:33.943+0000 7ff46d2d3640 -1 osd.3 0 log_to_monitors true 2024-09-16T00:34:34.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:34 smithi102 ceph-mon[35364]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:34:34.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:34 smithi031 ceph-mon[30956]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:34:34.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:34 smithi081 ceph-mon[35539]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:34:35.372 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:34:35.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_3 2024-09-16T00:34:35.653 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:34:35.675 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:35.675 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:34:35.675 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:34:35.675 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:34:35.675 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:35.675 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.676 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.676 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:34:35.676 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:34:35.676 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.676 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:35.676 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:35.676 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:35.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:34:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:35.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:35.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:35.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 -- 172.21.15.81:0/3558905972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 msgr2=0x7f7ba0104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3558905972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 0x7f7ba0104a50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009900 tx=0x7f7b9c02a410 comp rx=0 tx=0).stop 2024-09-16T00:34:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 -- 172.21.15.81:0/3558905972 shutdown_connections 2024-09-16T00:34:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3558905972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba01078c0 0x7f7ba0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3558905972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ba0104f90 0x7f7ba0107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3558905972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 0x7f7ba0104a50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 -- 172.21.15.81:0/3558905972 >> 172.21.15.81:0/3558905972 conn(0x7f7ba00fc280 msgr2=0x7f7ba00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 -- 172.21.15.81:0/3558905972 shutdown_connections 2024-09-16T00:34:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 -- 172.21.15.81:0/3558905972 wait complete. 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 Processor -- start 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 -- start start 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 0x7f7ba0101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ba0104f90 0x7f7ba0100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.141+0000 7f7ba861d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba01078c0 0x7f7ba0100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba861d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010d870 con 0x7f7ba01078c0 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba861d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010d6f0 con 0x7f7ba0102660 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba861d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010d9f0 con 0x7f7ba0104f90 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba63b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 0x7f7ba0101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba63b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 0x7f7ba0101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52496/0 (socket says 172.21.15.81:52496) 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba63b9640 1 -- 172.21.15.81:0/3414596341 learned_addr learned my addr 172.21.15.81:0/3414596341 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-09-16T00:34:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba6bba640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba01078c0 0x7f7ba0100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba5bb8640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ba0104f90 0x7f7ba0100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba63b9640 1 -- 172.21.15.81:0/3414596341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ba0104f90 msgr2=0x7f7ba0100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba63b9640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ba0104f90 0x7f7ba0100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba63b9640 1 -- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba01078c0 msgr2=0x7f7ba0100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba63b9640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba01078c0 0x7f7ba0100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba63b9640 1 -- 172.21.15.81:0/3414596341 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba0101410 con 0x7f7ba0102660 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba5bb8640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ba0104f90 0x7f7ba0100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba6bba640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba01078c0 0x7f7ba0100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba63b9640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 0x7f7ba0101fc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009230 tx=0x7f7b9c031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7b977fe640 1 -- 172.21.15.81:0/3414596341 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c03f030 con 0x7f7ba0102660 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba01a2050 con 0x7f7ba0102660 2024-09-16T00:34:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7b977fe640 1 -- 172.21.15.81:0/3414596341 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c003ad0 con 0x7f7ba0102660 2024-09-16T00:34:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7b977fe640 1 -- 172.21.15.81:0/3414596341 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c035070 con 0x7f7ba0102660 2024-09-16T00:34:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.142+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba01a2550 con 0x7f7ba0102660 2024-09-16T00:34:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.143+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba00fdd70 con 0x7f7ba0102660 2024-09-16T00:34:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.145+0000 7f7b977fe640 1 -- 172.21.15.81:0/3414596341 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7b9c042030 con 0x7f7ba0102660 2024-09-16T00:34:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.145+0000 7f7b977fe640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b8c0a3240 0x7f7b8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.145+0000 7f7b977fe640 1 -- 172.21.15.81:0/3414596341 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f7b9c0eebb0 con 0x7f7ba0102660 2024-09-16T00:34:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.145+0000 7f7ba5bb8640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b8c0a3240 0x7f7b8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.146+0000 7f7ba5bb8640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b8c0a3240 0x7f7b8c0a5700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b90005d00 tx=0x7f7b90005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.150+0000 7f7b977fe640 1 -- 172.21.15.81:0/3414596341 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b9c0bcc10 con 0x7f7ba0102660 2024-09-16T00:34:36.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:36.331+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7ba0062630 con 0x7f7b8c0a3240 2024-09-16T00:34:36.538 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3[49114]: 2024-09-16T00:34:36.181+0000 7ff469256640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T00:34:36.538 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3[49114]: 2024-09-16T00:34:36.186+0000 7ff461076640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:34:36.684 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi031 2024-09-16T00:34:36.684 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-16T00:34:36.684 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:34:36.684 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:34:36.684 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] boot 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T00:34:36.685 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:36 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:36.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi031 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:34:36.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:34:36.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:36.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] boot 2024-09-16T00:34:36.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T00:34:36.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:36 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:36.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi031 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: from='osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219]' entity='osd.3' 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: osd.3 [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] boot 2024-09-16T00:34:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T00:34:36.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:36 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:34:37.721 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:37 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:34:37.721 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:37 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:34:37.721 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:37 smithi081 ceph-mon[35539]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:34:37.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:37 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:34:37.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:37 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:34:37.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:37 smithi102 ceph-mon[35364]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:34:37.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:37 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:34:37.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:37 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:34:37.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:37 smithi031 ceph-mon[30956]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:34:38.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:38 smithi081 ceph-mon[35539]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:38.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:38 smithi081 ceph-mon[35539]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:34:38.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:38 smithi031 ceph-mon[30956]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:38.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:38 smithi031 ceph-mon[30956]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:34:38.933 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:38 smithi102 ceph-mon[35364]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:38.933 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:38 smithi102 ceph-mon[35364]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:34:39.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:39 smithi031 ceph-mon[30956]: from='client.? 172.21.15.81:0/3327527178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2942a206-0e2f-4a98-99d3-efc5325578f3"}]: dispatch 2024-09-16T00:34:39.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:39 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2942a206-0e2f-4a98-99d3-efc5325578f3"}]: dispatch 2024-09-16T00:34:39.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:39 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2942a206-0e2f-4a98-99d3-efc5325578f3"}]': finished 2024-09-16T00:34:39.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:39 smithi031 ceph-mon[30956]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T00:34:39.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:39 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:34:39.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:39 smithi031 ceph-mon[30956]: from='client.? 172.21.15.81:0/2537046596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:34:39.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:39 smithi031 ceph-mon[30956]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T00:34:39.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:39 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:34:39.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:39 smithi081 ceph-mon[35539]: from='client.? 172.21.15.81:0/3327527178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2942a206-0e2f-4a98-99d3-efc5325578f3"}]: dispatch 2024-09-16T00:34:39.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:39 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2942a206-0e2f-4a98-99d3-efc5325578f3"}]: dispatch 2024-09-16T00:34:39.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:39 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2942a206-0e2f-4a98-99d3-efc5325578f3"}]': finished 2024-09-16T00:34:39.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:39 smithi081 ceph-mon[35539]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T00:34:39.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:34:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:39 smithi081 ceph-mon[35539]: from='client.? 172.21.15.81:0/2537046596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:34:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:39 smithi081 ceph-mon[35539]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T00:34:39.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:34:40.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:39 smithi102 ceph-mon[35364]: from='client.? 172.21.15.81:0/3327527178' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2942a206-0e2f-4a98-99d3-efc5325578f3"}]: dispatch 2024-09-16T00:34:40.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:39 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2942a206-0e2f-4a98-99d3-efc5325578f3"}]: dispatch 2024-09-16T00:34:40.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:39 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2942a206-0e2f-4a98-99d3-efc5325578f3"}]': finished 2024-09-16T00:34:40.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:39 smithi102 ceph-mon[35364]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T00:34:40.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:39 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:34:40.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:39 smithi102 ceph-mon[35364]: from='client.? 172.21.15.81:0/2537046596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:34:40.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:39 smithi102 ceph-mon[35364]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T00:34:40.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:39 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:34:40.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:40 smithi031 ceph-mon[30956]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:40.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:40 smithi031 ceph-mon[30956]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:34:40.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:40 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:34:40.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:40 smithi081 ceph-mon[35539]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:40.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:40 smithi081 ceph-mon[35539]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:34:40.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:40 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:34:41.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:40 smithi102 ceph-mon[35364]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:41.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:40 smithi102 ceph-mon[35364]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:34:41.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:40 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:34:42.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:42 smithi081 ceph-mon[35539]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:34:43.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:42 smithi102 ceph-mon[35364]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:34:43.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:42 smithi031 ceph-mon[30956]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:34:44.984 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:44 smithi081 ceph-mon[35539]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:34:45.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:44 smithi102 ceph-mon[35364]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:34:45.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:44 smithi031 ceph-mon[30956]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:34:46.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:46 smithi081 ceph-mon[35539]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-16T00:34:47.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:46 smithi102 ceph-mon[35364]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-16T00:34:47.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:46 smithi031 ceph-mon[30956]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-16T00:34:47.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:34:47.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:47.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:47 smithi081 ceph-mon[35539]: Deploying daemon osd.4 on smithi081 2024-09-16T00:34:48.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:34:48.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:48.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:47 smithi102 ceph-mon[35364]: Deploying daemon osd.4 on smithi081 2024-09-16T00:34:48.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:34:48.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:48.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:47 smithi031 ceph-mon[30956]: Deploying daemon osd.4 on smithi081 2024-09-16T00:34:48.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:48 smithi102 ceph-mon[35364]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-16T00:34:48.960 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:48 smithi081 ceph-mon[35539]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-16T00:34:49.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:48 smithi031 ceph-mon[30956]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-16T00:34:50.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:50 smithi081 ceph-mon[35539]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:34:51.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:50 smithi102 ceph-mon[35364]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:34:51.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:50 smithi031 ceph-mon[30956]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:34:52.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:52 smithi102 ceph-mon[35364]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:34:52.950 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:52 smithi081 ceph-mon[35539]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:34:53.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:52 smithi031 ceph-mon[30956]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:34:53.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:53.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:53.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:54.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:54.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:54.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:54.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:54.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:54.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:55 smithi081 ceph-mon[35539]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:55 smithi102 ceph-mon[35364]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:55.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:55 smithi031 ceph-mon[30956]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:56.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:56.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:56.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:56.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:56.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:56.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:56 smithi081 ceph-mon[35539]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:56.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:56.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:56.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:56.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:56.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:56.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:56 smithi031 ceph-mon[30956]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:57.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:57.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:57.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:57.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:57.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:57.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:56 smithi102 ceph-mon[35364]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:57.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.157+0000 7f7b977fe640 1 -- 172.21.15.81:0/3414596341 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7ba0062630 con 0x7f7b8c0a3240 2024-09-16T00:34:57.158 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 4 on host 'smithi081' 2024-09-16T00:34:57.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b8c0a3240 msgr2=0x7f7b8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:57.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b8c0a3240 0x7f7b8c0a5700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b90005d00 tx=0x7f7b90005c70 comp rx=0 tx=0).stop 2024-09-16T00:34:57.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 msgr2=0x7f7ba0101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:57.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 0x7f7ba0101fc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009230 tx=0x7f7b9c031b60 comp rx=0 tx=0).stop 2024-09-16T00:34:57.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 shutdown_connections 2024-09-16T00:34:57.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba01078c0 0x7f7ba0100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b8c0a3240 0x7f7b8c0a5700 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ba0104f90 0x7f7ba0100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 --2- 172.21.15.81:0/3414596341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ba0102660 0x7f7ba0101fc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 >> 172.21.15.81:0/3414596341 conn(0x7f7ba00fc280 msgr2=0x7f7ba0103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:57.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 shutdown_connections 2024-09-16T00:34:57.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:34:57.159+0000 7f7ba861d640 1 -- 172.21.15.81:0/3414596341 wait complete. 2024-09-16T00:34:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:57.810 DEBUG:teuthology.orchestra.run.smithi081:osd.4> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T00:34:57.813 INFO:tasks.cephadm:Deploying osd.5 on smithi102 with /dev/vg_nvme/lv_3... 2024-09-16T00:34:57.813 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:34:57.839 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:57.839 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:57.840 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:57.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:57.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:57.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:34:58.060 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:34:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:58 smithi102 ceph-mon[35364]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:58.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:58 smithi031 ceph-mon[30956]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:58.964 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:58 smithi081 ceph-mon[35539]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:34:59.247 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:34:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4[46602]: 2024-09-16T00:34:59.037+0000 7f7e0e861640 -1 osd.4 0 log_to_monitors true 2024-09-16T00:34:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:59 smithi102 ceph-mon[35364]: from='osd.4 [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:34:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:34:59 smithi102 ceph-mon[35364]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:34:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:59 smithi031 ceph-mon[30956]: from='osd.4 [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:34:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:34:59 smithi031 ceph-mon[30956]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:34:59.921 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:59 smithi081 ceph-mon[35539]: from='osd.4 [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:34:59.921 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:34:59 smithi081 ceph-mon[35539]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:35:00.669 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:35:00.694 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2024-09-16T00:35:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:00 smithi081 ceph-mon[35539]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:35:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:00 smithi081 ceph-mon[35539]: from='osd.4 [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:35:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:00 smithi081 ceph-mon[35539]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:35:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:00 smithi081 ceph-mon[35539]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:35:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:00 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:00 smithi081 ceph-mon[35539]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:35:00.747 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:35:00 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4[46602]: 2024-09-16T00:35:00.611+0000 7f7e0a7e4640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T00:35:00.748 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:35:00 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4[46602]: 2024-09-16T00:35:00.616+0000 7f7e0660c640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:35:00.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:00 smithi031 ceph-mon[30956]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:35:00.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:00 smithi031 ceph-mon[30956]: from='osd.4 [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:35:00.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:00 smithi031 ceph-mon[30956]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:35:00.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:00 smithi031 ceph-mon[30956]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:35:00.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:00 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:00.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:00 smithi031 ceph-mon[30956]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:35:00.980 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:35:01.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:00 smithi102 ceph-mon[35364]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:35:01.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:00 smithi102 ceph-mon[35364]: from='osd.4 [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:35:01.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:00 smithi102 ceph-mon[35364]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:35:01.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:00 smithi102 ceph-mon[35364]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:35:01.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:00 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:01.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:00 smithi102 ceph-mon[35364]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.469+0000 7f4e75a57640 1 -- 172.21.15.102:0/1010054454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e70102e00 msgr2=0x7f4e70103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.469+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1010054454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e70102e00 0x7f4e70103280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009900 tx=0x7f4e6402a410 comp rx=0 tx=0).stop 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.470+0000 7f4e75a57640 1 -- 172.21.15.102:0/1010054454 shutdown_connections 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.470+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1010054454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e701037c0 0x7f4e7010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.470+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1010054454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e70102e00 0x7f4e70103280 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.470+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1010054454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70101c00 0x7f4e70102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.470+0000 7f4e75a57640 1 -- 172.21.15.102:0/1010054454 >> 172.21.15.102:0/1010054454 conn(0x7f4e700fd390 msgr2=0x7f4e700ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.470+0000 7f4e75a57640 1 -- 172.21.15.102:0/1010054454 shutdown_connections 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.470+0000 7f4e75a57640 1 -- 172.21.15.102:0/1010054454 wait complete. 2024-09-16T00:35:01.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.470+0000 7f4e75a57640 1 Processor -- start 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.470+0000 7f4e75a57640 1 -- start start 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e75a57640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70101c00 0x7f4e70197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e75a57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102e00 0x7f4e701983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e75a57640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e701037c0 0x7f4e7019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e75a57640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010d860 con 0x7f4e70102e00 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e75a57640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010d6e0 con 0x7f4e70101c00 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e75a57640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010d9e0 con 0x7f4e701037c0 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e701037c0 0x7f4e7019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e701037c0 0x7f4e7019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46334/0 (socket says 172.21.15.102:46334) 2024-09-16T00:35:01.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6f7fe640 1 -- 172.21.15.102:0/1370068218 learned_addr learned my addr 172.21.15.102:0/1370068218 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6e7fc640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102e00 0x7f4e701983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6effd640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70101c00 0x7f4e70197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6f7fe640 1 -- 172.21.15.102:0/1370068218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70101c00 msgr2=0x7f4e70197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6f7fe640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70101c00 0x7f4e70197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6f7fe640 1 -- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102e00 msgr2=0x7f4e701983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6f7fe640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102e00 0x7f4e701983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6f7fe640 1 -- 172.21.15.102:0/1370068218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7019fab0 con 0x7f4e701037c0 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e6f7fe640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e701037c0 0x7f4e7019f440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800d950 tx=0x7f4e6800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e74a55640 1 -- 172.21.15.102:0/1370068218 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68017030 con 0x7f4e701037c0 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e74a55640 1 -- 172.21.15.102:0/1370068218 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e68004020 con 0x7f4e701037c0 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7019fda0 con 0x7f4e701037c0 2024-09-16T00:35:01.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e74a55640 1 -- 172.21.15.102:0/1370068218 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68003c00 con 0x7f4e701037c0 2024-09-16T00:35:01.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.471+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701a0250 con 0x7f4e701037c0 2024-09-16T00:35:01.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.472+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e70109050 con 0x7f4e701037c0 2024-09-16T00:35:01.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.473+0000 7f4e74a55640 1 -- 172.21.15.102:0/1370068218 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4e6801a070 con 0x7f4e701037c0 2024-09-16T00:35:01.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.474+0000 7f4e74a55640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4e580a32f0 0x7f4e580a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:01.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.474+0000 7f4e74a55640 1 -- 172.21.15.102:0/1370068218 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f4e680cdfb0 con 0x7f4e701037c0 2024-09-16T00:35:01.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.474+0000 7f4e6effd640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4e580a32f0 0x7f4e580a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:01.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.475+0000 7f4e6effd640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4e580a32f0 0x7f4e580a57b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005c40 tx=0x7f4e60005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:01.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.478+0000 7f4e74a55640 1 -- 172.21.15.102:0/1370068218 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e68021070 con 0x7f4e701037c0 2024-09-16T00:35:01.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:01.657+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4e70062630 con 0x7f4e580a32f0 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi081 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi081 to 8544M 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:01.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:01.659 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:01.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi081 2024-09-16T00:35:01.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:01.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:35:01.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:01.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:35:01.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:01.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:01.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:01.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi081 to 8544M 2024-09-16T00:35:01.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:01.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:01.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:01.880 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi081 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi081 to 8544M 2024-09-16T00:35:02.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:02.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:02.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:02.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:02.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: osd.4 [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] boot 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:35:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:02.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:35:02.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:35:02.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:35:02.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:02.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: osd.4 [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] boot 2024-09-16T00:35:02.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:35:02.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:02.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:02.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:35:02.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:35:02.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:03.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:35:03.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:35:03.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:35:03.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:03.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: osd.4 [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] boot 2024-09-16T00:35:03.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:35:03.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:35:03.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:03.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:35:03.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:35:03.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:03.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:03 smithi081 ceph-mon[35539]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:35:04.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:03 smithi102 ceph-mon[35364]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:35:04.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:03 smithi031 ceph-mon[30956]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:35:04.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:04 smithi081 ceph-mon[35539]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:04.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:04 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/1530205689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "893456fd-1776-4a46-b16c-4590c38b3ee8"}]: dispatch 2024-09-16T00:35:04.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:04 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "893456fd-1776-4a46-b16c-4590c38b3ee8"}]: dispatch 2024-09-16T00:35:04.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:04 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "893456fd-1776-4a46-b16c-4590c38b3ee8"}]': finished 2024-09-16T00:35:04.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:04 smithi081 ceph-mon[35539]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:35:04.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:04.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:04 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/1984428595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:35:05.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:04 smithi102 ceph-mon[35364]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:05.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:04 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/1530205689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "893456fd-1776-4a46-b16c-4590c38b3ee8"}]: dispatch 2024-09-16T00:35:05.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:04 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "893456fd-1776-4a46-b16c-4590c38b3ee8"}]: dispatch 2024-09-16T00:35:05.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:04 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "893456fd-1776-4a46-b16c-4590c38b3ee8"}]': finished 2024-09-16T00:35:05.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:04 smithi102 ceph-mon[35364]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:35:05.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:05.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:04 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/1984428595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:35:05.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:04 smithi031 ceph-mon[30956]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:05.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:04 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/1530205689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "893456fd-1776-4a46-b16c-4590c38b3ee8"}]: dispatch 2024-09-16T00:35:05.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:04 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "893456fd-1776-4a46-b16c-4590c38b3ee8"}]: dispatch 2024-09-16T00:35:05.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:04 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "893456fd-1776-4a46-b16c-4590c38b3ee8"}]': finished 2024-09-16T00:35:05.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:04 smithi031 ceph-mon[30956]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:35:05.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:05.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:04 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/1984428595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:35:06.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:06 smithi031 ceph-mon[30956]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:06.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:06 smithi081 ceph-mon[35539]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:07.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:06 smithi102 ceph-mon[35364]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:08.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:08 smithi102 ceph-mon[35364]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:08.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:08 smithi081 ceph-mon[35539]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:09.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:08 smithi031 ceph-mon[30956]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:10.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:10 smithi081 ceph-mon[35539]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:11.034 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:10 smithi102 ceph-mon[35364]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:11.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:10 smithi031 ceph-mon[30956]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:12.953 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:12 smithi102 ceph-mon[35364]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:12.953 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:35:12.953 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:12.953 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:12 smithi102 ceph-mon[35364]: Deploying daemon osd.5 on smithi102 2024-09-16T00:35:12.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:12 smithi081 ceph-mon[35539]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:12.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:35:12.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:12.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:12 smithi081 ceph-mon[35539]: Deploying daemon osd.5 on smithi102 2024-09-16T00:35:13.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:12 smithi031 ceph-mon[30956]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:13.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:35:13.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:13.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:12 smithi031 ceph-mon[30956]: Deploying daemon osd.5 on smithi102 2024-09-16T00:35:14.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:14 smithi102 ceph-mon[35364]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:14.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:14 smithi081 ceph-mon[35539]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:15.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:14 smithi031 ceph-mon[30956]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:16.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:16 smithi081 ceph-mon[35539]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:17.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:16 smithi102 ceph-mon[35364]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:17.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:16 smithi031 ceph-mon[30956]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:18.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:18 smithi102 ceph-mon[35364]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:18 smithi081 ceph-mon[35539]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:19.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:18 smithi031 ceph-mon[30956]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:19 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:19 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:19 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:19.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:19 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:19.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:19 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:19.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:19 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:20.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:20.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:20.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:19 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:20.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:20 smithi102 ceph-mon[35364]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:20.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:20 smithi081 ceph-mon[35539]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:21.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:20 smithi031 ceph-mon[30956]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:22.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:22 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:22.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:22 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:22.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:22 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:22.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:22 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:22.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:22 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 5 on host 'smithi102' 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.751+0000 7f4e74a55640 1 -- 172.21.15.102:0/1370068218 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4e70062630 con 0x7f4e580a32f0 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.753+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4e580a32f0 msgr2=0x7f4e580a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.753+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4e580a32f0 0x7f4e580a57b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005c40 tx=0x7f4e60005bd0 comp rx=0 tx=0).stop 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e701037c0 msgr2=0x7f4e7019f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e701037c0 0x7f4e7019f440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800d950 tx=0x7f4e6800de10 comp rx=0 tx=0).stop 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 shutdown_connections 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4e580a32f0 0x7f4e580a57b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e701037c0 0x7f4e7019f440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102e00 0x7f4e701983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 --2- 172.21.15.102:0/1370068218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70101c00 0x7f4e70197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 >> 172.21.15.102:0/1370068218 conn(0x7f4e700fd390 msgr2=0x7f4e70106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 shutdown_connections 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:35:22.754+0000 7f4e75a57640 1 -- 172.21.15.102:0/1370068218 wait complete. 2024-09-16T00:35:22.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:22.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:22.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:22.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:22.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:22.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:22 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:22.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:22 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:22.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:22 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:22.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:22 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:22.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:22 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:23.417 DEBUG:teuthology.orchestra.run.smithi102:osd.5> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T00:35:23.420 INFO:tasks.cephadm:Deploying osd.6 on smithi031 with /dev/vg_nvme/lv_2... 2024-09-16T00:35:23.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:35:23.687 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:23.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:23 smithi102 ceph-mon[35364]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:23.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:23.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:23.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:23.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:23 smithi031 ceph-mon[30956]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:23.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:23.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:23.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:23.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:23 smithi081 ceph-mon[35539]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:23.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:23.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:23.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:24.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:24 smithi102 ceph-mon[35364]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:24.846 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:24 smithi031 ceph-mon[30956]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:24.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:24 smithi081 ceph-mon[35539]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:25.300 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:35:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:35:25.029+0000 7f4ea5d32640 -1 osd.5 0 log_to_monitors true 2024-09-16T00:35:25.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:25 smithi031 ceph-mon[30956]: from='osd.5 [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:35:25.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:25 smithi031 ceph-mon[30956]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:35:25.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:25 smithi102 ceph-mon[35364]: from='osd.5 [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:35:25.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:25 smithi102 ceph-mon[35364]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:35:25.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:25 smithi081 ceph-mon[35539]: from='osd.5 [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:35:25.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:25 smithi081 ceph-mon[35539]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:35:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:35:26.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-09-16T00:35:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:26.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:35:26.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='osd.5 [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:35:26.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:35:26.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:26.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:35:26.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:26.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi102 2024-09-16T00:35:26.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:26.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:26.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:26.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:26.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi102 to 8544M 2024-09-16T00:35:26.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:26.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:26.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:26.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:26.654 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:26 smithi031 ceph-mon[30956]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:35:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.847+0000 7fd8904cd640 1 -- 172.21.15.31:0/601040175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 msgr2=0x7fd888100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.847+0000 7fd8904cd640 1 --2- 172.21.15.31:0/601040175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 0x7fd888100f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd884009900 tx=0x7fd88402a410 comp rx=0 tx=0).stop 2024-09-16T00:35:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.848+0000 7fd8904cd640 1 -- 172.21.15.31:0/601040175 shutdown_connections 2024-09-16T00:35:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.848+0000 7fd8904cd640 1 --2- 172.21.15.31:0/601040175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8881026c0 0x7fd888108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.848+0000 7fd8904cd640 1 --2- 172.21.15.31:0/601040175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd888101d00 0x7fd888102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.848+0000 7fd8904cd640 1 --2- 172.21.15.31:0/601040175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 0x7fd888100f00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.848+0000 7fd8904cd640 1 -- 172.21.15.31:0/601040175 >> 172.21.15.31:0/601040175 conn(0x7fd8880fc290 msgr2=0x7fd8880fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.848+0000 7fd8904cd640 1 -- 172.21.15.31:0/601040175 shutdown_connections 2024-09-16T00:35:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.848+0000 7fd8904cd640 1 -- 172.21.15.31:0/601040175 wait complete. 2024-09-16T00:35:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.848+0000 7fd8904cd640 1 Processor -- start 2024-09-16T00:35:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.848+0000 7fd8904cd640 1 -- start start 2024-09-16T00:35:26.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd8904cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 0x7fd88819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:26.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd8904cd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd888101d00 0x7fd88819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:26.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd8904cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8881026c0 0x7fd8881a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:26.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd8904cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd88810c760 con 0x7fd888100b00 2024-09-16T00:35:26.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd8904cd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd88810c5e0 con 0x7fd888101d00 2024-09-16T00:35:26.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd8904cd640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd88810c8e0 con 0x7fd8881026c0 2024-09-16T00:35:26.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88e269640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 0x7fd88819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:26.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88e269640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 0x7fd88819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45272/0 (socket says 172.21.15.31:45272) 2024-09-16T00:35:26.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88e269640 1 -- 172.21.15.31:0/1131682937 learned_addr learned my addr 172.21.15.31:0/1131682937 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:35:26.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88ea6a640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8881026c0 0x7fd8881a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:26.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88da68640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd888101d00 0x7fd88819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88e269640 1 -- 172.21.15.31:0/1131682937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8881026c0 msgr2=0x7fd8881a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88e269640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8881026c0 0x7fd8881a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88e269640 1 -- 172.21.15.31:0/1131682937 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd888101d00 msgr2=0x7fd88819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88e269640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd888101d00 0x7fd88819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88e269640 1 -- 172.21.15.31:0/1131682937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8881a1d40 con 0x7fd888100b00 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88ea6a640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8881026c0 0x7fd8881a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd88e269640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 0x7fd88819a110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd884031610 tx=0x7fd884004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.849+0000 7fd87f7fe640 1 -- 172.21.15.31:0/1131682937 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88403f030 con 0x7fd888100b00 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.850+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8881a1fd0 con 0x7fd888100b00 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.850+0000 7fd87f7fe640 1 -- 172.21.15.31:0/1131682937 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884003ad0 con 0x7fd888100b00 2024-09-16T00:35:26.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.850+0000 7fd87f7fe640 1 -- 172.21.15.31:0/1131682937 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884035070 con 0x7fd888100b00 2024-09-16T00:35:26.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.850+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881a23d0 con 0x7fd888100b00 2024-09-16T00:35:26.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.851+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8881a26c0 con 0x7fd888100b00 2024-09-16T00:35:26.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.852+0000 7fd87f7fe640 1 -- 172.21.15.31:0/1131682937 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd884042030 con 0x7fd888100b00 2024-09-16T00:35:26.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.853+0000 7fd87f7fe640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd8740a3270 0x7fd8740a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:26.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.853+0000 7fd88da68640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd8740a3270 0x7fd8740a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:26.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.853+0000 7fd87f7fe640 1 -- 172.21.15.31:0/1131682937 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7fd8840ef250 con 0x7fd888100b00 2024-09-16T00:35:26.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.853+0000 7fd88da68640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd8740a3270 0x7fd8740a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd88819b630 tx=0x7fd878005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:26.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:26.856+0000 7fd87f7fe640 1 -- 172.21.15.31:0/1131682937 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8840bc860 con 0x7fd888100b00 2024-09-16T00:35:26.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:35:26.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='osd.5 [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:35:26.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:35:26.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:26.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:35:26.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi102 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi102 to 8544M 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:26.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:26 smithi081 ceph-mon[35539]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:35:27.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:27.043+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd88806af50 con 0x7fd8740a3270 2024-09-16T00:35:27.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:35:27.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='osd.5 [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:35:27.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:35:27.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi102 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi102 to 8544M 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:27.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:26 smithi102 ceph-mon[35364]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:35:27.549 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:35:27.193+0000 7f4ea24c9640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T00:35:27.549 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:35:27.199+0000 7f4e99ad5640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:35:27.705 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:35:27.705 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:27.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:27.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:27.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:35:27.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:35:27.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:27.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: from='osd.5 ' entity='osd.5' 2024-09-16T00:35:27.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: osd.5 [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] boot 2024-09-16T00:35:27.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:35:27.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:27 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:27.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:35:27.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:27.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:27.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:27.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:35:27.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:35:27.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:27.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: from='osd.5 ' entity='osd.5' 2024-09-16T00:35:27.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: osd.5 [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] boot 2024-09-16T00:35:27.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:35:27.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:27 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:28.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:35:28.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:28.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:28.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:28.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:35:28.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:35:28.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:28.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: from='osd.5 ' entity='osd.5' 2024-09-16T00:35:28.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: osd.5 [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] boot 2024-09-16T00:35:28.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:35:28.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:27 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:35:28.739 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:28 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:35:28.739 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:28 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:35:28.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:28 smithi031 ceph-mon[30956]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:28.740 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:28 smithi031 ceph-mon[30956]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:35:28.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:28 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:35:28.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:28 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:35:28.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:28 smithi081 ceph-mon[35539]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:28.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:28 smithi081 ceph-mon[35539]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:35:29.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:28 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:35:29.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:28 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:35:29.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:28 smithi102 ceph-mon[35364]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:29.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:28 smithi102 ceph-mon[35364]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:35:29.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:29 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2740291760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "749c866d-0f7f-4a78-ac75-5728ed7d49ca"}]: dispatch 2024-09-16T00:35:29.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:29 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2740291760' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "749c866d-0f7f-4a78-ac75-5728ed7d49ca"}]': finished 2024-09-16T00:35:29.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:29 smithi031 ceph-mon[30956]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T00:35:29.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:29.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:29 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2740291760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "749c866d-0f7f-4a78-ac75-5728ed7d49ca"}]: dispatch 2024-09-16T00:35:29.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:29 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2740291760' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "749c866d-0f7f-4a78-ac75-5728ed7d49ca"}]': finished 2024-09-16T00:35:29.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:29 smithi081 ceph-mon[35539]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T00:35:29.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:30.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:29 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2740291760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "749c866d-0f7f-4a78-ac75-5728ed7d49ca"}]: dispatch 2024-09-16T00:35:30.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:29 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2740291760' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "749c866d-0f7f-4a78-ac75-5728ed7d49ca"}]': finished 2024-09-16T00:35:30.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:29 smithi102 ceph-mon[35364]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T00:35:30.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:30.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:30 smithi031 ceph-mon[30956]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:30.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:30 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2452668835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:35:30.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:30 smithi081 ceph-mon[35539]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:30.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:30 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2452668835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:35:31.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:30 smithi102 ceph-mon[35364]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:31.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:30 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2452668835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:35:31.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:31 smithi031 ceph-mon[30956]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:35:31.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:31.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:31 smithi081 ceph-mon[35539]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:35:31.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:32.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:31 smithi102 ceph-mon[35364]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:35:32.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:32.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:32 smithi031 ceph-mon[30956]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:35:32.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:32 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:32.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:32 smithi031 ceph-mon[30956]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:35:32.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:32 smithi081 ceph-mon[35539]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:35:32.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:32 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:32.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:32 smithi081 ceph-mon[35539]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:35:33.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:32 smithi102 ceph-mon[35364]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:35:33.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:32 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:33.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:32 smithi102 ceph-mon[35364]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:35:34.810 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:34 smithi031 ceph-mon[30956]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:35:34.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:34 smithi081 ceph-mon[35539]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:35:35.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:34 smithi102 ceph-mon[35364]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:35:36.859 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:36 smithi031 ceph-mon[30956]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-16T00:35:36.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:36 smithi081 ceph-mon[35539]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-16T00:35:37.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:36 smithi102 ceph-mon[35364]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-16T00:35:38.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:38 smithi081 ceph-mon[35539]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:35:38.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:35:38.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:38.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:38 smithi081 ceph-mon[35539]: Deploying daemon osd.6 on smithi031 2024-09-16T00:35:39.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:38 smithi102 ceph-mon[35364]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:35:39.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:35:39.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:39.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:38 smithi102 ceph-mon[35364]: Deploying daemon osd.6 on smithi031 2024-09-16T00:35:39.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:38 smithi031 ceph-mon[30956]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:35:39.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:35:39.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:39.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:38 smithi031 ceph-mon[30956]: Deploying daemon osd.6 on smithi031 2024-09-16T00:35:40.672 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:40 smithi031 ceph-mon[30956]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T00:35:40.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:40 smithi081 ceph-mon[35539]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T00:35:41.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:40 smithi102 ceph-mon[35364]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T00:35:42.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:42 smithi031 ceph-mon[30956]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:42.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:42 smithi081 ceph-mon[35539]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:43.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:42 smithi102 ceph-mon[35364]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:44.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:44 smithi031 ceph-mon[30956]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:44.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:44.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:44.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:44.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:44 smithi081 ceph-mon[35539]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:44.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:44.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:44.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:45.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:44 smithi102 ceph-mon[35364]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:45.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:45.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:45.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:46.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:46 smithi081 ceph-mon[35539]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:46.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:46.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:46.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:46.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:46 smithi031 ceph-mon[30956]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:47.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:47.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:47.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:46 smithi102 ceph-mon[35364]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:47.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:47.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:47.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.654+0000 7fd87f7fe640 1 -- 172.21.15.31:0/1131682937 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd88806af50 con 0x7fd8740a3270 2024-09-16T00:35:47.655 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 6 on host 'smithi031' 2024-09-16T00:35:47.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd8740a3270 msgr2=0x7fd8740a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:47.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd8740a3270 0x7fd8740a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd88819b630 tx=0x7fd878005f90 comp rx=0 tx=0).stop 2024-09-16T00:35:47.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 msgr2=0x7fd88819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:47.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 0x7fd88819a110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd884031610 tx=0x7fd884004270 comp rx=0 tx=0).stop 2024-09-16T00:35:47.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 shutdown_connections 2024-09-16T00:35:47.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd8740a3270 0x7fd8740a5730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8881026c0 0x7fd8881a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd888101d00 0x7fd88819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 --2- 172.21.15.31:0/1131682937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd888100b00 0x7fd88819a110 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.656+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 >> 172.21.15.31:0/1131682937 conn(0x7fd8880fc290 msgr2=0x7fd88806a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:47.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.657+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 shutdown_connections 2024-09-16T00:35:47.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:35:47.657+0000 7fd8904cd640 1 -- 172.21.15.31:0/1131682937 wait complete. 2024-09-16T00:35:47.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:47.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:47.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:47.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:48.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:48.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:48.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:48.273 DEBUG:teuthology.orchestra.run.smithi031:osd.6> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T00:35:48.275 INFO:tasks.cephadm:Deploying osd.7 on smithi081 with /dev/vg_nvme/lv_2... 2024-09-16T00:35:48.276 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:35:48.537 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:35:48.815 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:48 smithi031 ceph-mon[30956]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:48.983 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:48 smithi081 ceph-mon[35539]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:49.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:48 smithi102 ceph-mon[35364]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:49.686 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:35:49 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:35:49.427+0000 7f5423d9f640 -1 osd.6 0 log_to_monitors true 2024-09-16T00:35:49.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:49 smithi031 ceph-mon[30956]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:35:49.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:49 smithi081 ceph-mon[35539]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:35:50.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:49 smithi102 ceph-mon[35364]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:35:50.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:50 smithi031 ceph-mon[30956]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:50.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:50 smithi031 ceph-mon[30956]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:35:50.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:50 smithi031 ceph-mon[30956]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:35:50.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:50 smithi031 ceph-mon[30956]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:35:50.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:50.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:35:50 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:35:50.696+0000 7f541fd22640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T00:35:50.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:35:50 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:35:50.705+0000 7f541bb4a640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:35:50.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:50 smithi081 ceph-mon[35539]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:50.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:50 smithi081 ceph-mon[35539]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:35:50.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:50 smithi081 ceph-mon[35539]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:35:50.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:50 smithi081 ceph-mon[35539]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:35:50.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:51.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:50 smithi102 ceph-mon[35364]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:51.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:50 smithi102 ceph-mon[35364]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:35:51.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:50 smithi102 ceph-mon[35364]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:35:51.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:50 smithi102 ceph-mon[35364]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:35:51.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:51.232 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:35:51.259 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_2 2024-09-16T00:35:51.512 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:35:51.950 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:35:51.950 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T00:35:51.950 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:51.950 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:51.950 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi031 2024-09-16T00:35:51.950 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:51.950 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:51.950 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:51.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:51.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:51.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-16T00:35:51.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:51.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:51.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:51.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:51.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:51.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 -- 172.21.15.81:0/77199343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc101cd0 msgr2=0x7efefc102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:51.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 --2- 172.21.15.81:0/77199343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc101cd0 0x7efefc102150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efee4009900 tx=0x7efee402a410 comp rx=0 tx=0).stop 2024-09-16T00:35:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 -- 172.21.15.81:0/77199343 shutdown_connections 2024-09-16T00:35:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 --2- 172.21.15.81:0/77199343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efefc102690 0x7efefc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 --2- 172.21.15.81:0/77199343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc101cd0 0x7efefc102150 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 --2- 172.21.15.81:0/77199343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efefc100ad0 0x7efefc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 -- 172.21.15.81:0/77199343 >> 172.21.15.81:0/77199343 conn(0x7efefc0fc280 msgr2=0x7efefc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 -- 172.21.15.81:0/77199343 shutdown_connections 2024-09-16T00:35:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 -- 172.21.15.81:0/77199343 wait complete. 2024-09-16T00:35:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 Processor -- start 2024-09-16T00:35:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.958+0000 7eff01521640 1 -- start start 2024-09-16T00:35:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7eff01521640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc100ad0 0x7efefc104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7eff01521640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efefc101cd0 0x7efefc1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7eff01521640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efefc102690 0x7efefc1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7eff01521640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efefc10c750 con 0x7efefc101cd0 2024-09-16T00:35:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7eff01521640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efefc10c5d0 con 0x7efefc100ad0 2024-09-16T00:35:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7eff01521640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efefc10c8d0 con 0x7efefc102690 2024-09-16T00:35:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefaffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc100ad0 0x7efefc104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefaffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc100ad0 0x7efefc104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57924/0 (socket says 172.21.15.81:57924) 2024-09-16T00:35:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefaffd640 1 -- 172.21.15.81:0/2424726664 learned_addr learned my addr 172.21.15.81:0/2424726664 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-09-16T00:35:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefb7fe640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efefc102690 0x7efefc1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefa7fc640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efefc101cd0 0x7efefc1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefaffd640 1 -- 172.21.15.81:0/2424726664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efefc102690 msgr2=0x7efefc1035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefaffd640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efefc102690 0x7efefc1035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefaffd640 1 -- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efefc101cd0 msgr2=0x7efefc1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefaffd640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efefc101cd0 0x7efefc1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefaffd640 1 -- 172.21.15.81:0/2424726664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc103eb0 con 0x7efefc100ad0 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefb7fe640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efefc102690 0x7efefc1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efefaffd640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc100ad0 0x7efefc104a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efef400eaa0 tx=0x7efef400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efef3fff640 1 -- 172.21.15.81:0/2424726664 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4016070 con 0x7efefc100ad0 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc1a20f0 con 0x7efefc100ad0 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efef3fff640 1 -- 172.21.15.81:0/2424726664 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef4003e50 con 0x7efefc100ad0 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.959+0000 7efef3fff640 1 -- 172.21.15.81:0/2424726664 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef40116d0 con 0x7efefc100ad0 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.960+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc1a25b0 con 0x7efefc100ad0 2024-09-16T00:35:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.961+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efefc100ed0 con 0x7efefc100ad0 2024-09-16T00:35:51.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.962+0000 7efef3fff640 1 -- 172.21.15.81:0/2424726664 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efef4019070 con 0x7efefc100ad0 2024-09-16T00:35:51.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.962+0000 7efef3fff640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efee80a3240 0x7efee80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:51.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.963+0000 7efef3fff640 1 -- 172.21.15.81:0/2424726664 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7efef40cf190 con 0x7efefc100ad0 2024-09-16T00:35:51.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.963+0000 7efefa7fc640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efee80a3240 0x7efee80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:51.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.963+0000 7efefa7fc640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efee80a3240 0x7efee80a5700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efee4000c00 tx=0x7efee40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:51.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:51.966+0000 7efef3fff640 1 -- 172.21.15.81:0/2424726664 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efef409cbc0 con 0x7efefc100ad0 2024-09-16T00:35:52.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:35:52.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T00:35:52.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi031 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:52.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:52.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi031 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:35:52.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-16T00:35:52.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:52.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:52.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:52.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:35:52.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:52.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:35:52.141+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efefc06a770 con 0x7efee80a3240 2024-09-16T00:35:52.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:35:52.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:35:52.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:52.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] boot 2024-09-16T00:35:52.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T00:35:52.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:52.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:52.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:35:52.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:35:52.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:52 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:53.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:35:53.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:35:53.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:53.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] boot 2024-09-16T00:35:53.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T00:35:53.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:53.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:53.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:35:53.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:35:53.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:52 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:53.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:35:53.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:35:53.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:35:53.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: osd.6 [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] boot 2024-09-16T00:35:53.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T00:35:53.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:35:53.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:53.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:35:53.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:35:53.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:52 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:54.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:54 smithi081 ceph-mon[35539]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T00:35:54.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:54 smithi102 ceph-mon[35364]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T00:35:54.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:54 smithi031 ceph-mon[30956]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T00:35:55.501 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:55 smithi081 ceph-mon[35539]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:35:55.501 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:55 smithi081 ceph-mon[35539]: from='client.? 172.21.15.81:0/3255721000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f7eade8-dd2f-42ce-8b27-0c03f9172b79"}]: dispatch 2024-09-16T00:35:55.501 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:55 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f7eade8-dd2f-42ce-8b27-0c03f9172b79"}]: dispatch 2024-09-16T00:35:55.501 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:55 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f7eade8-dd2f-42ce-8b27-0c03f9172b79"}]': finished 2024-09-16T00:35:55.501 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:55 smithi081 ceph-mon[35539]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T00:35:55.502 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:35:55.502 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:55 smithi081 ceph-mon[35539]: from='client.? 172.21.15.81:0/980131825' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:35:55.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:55 smithi031 ceph-mon[30956]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:35:55.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:55 smithi031 ceph-mon[30956]: from='client.? 172.21.15.81:0/3255721000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f7eade8-dd2f-42ce-8b27-0c03f9172b79"}]: dispatch 2024-09-16T00:35:55.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:55 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f7eade8-dd2f-42ce-8b27-0c03f9172b79"}]: dispatch 2024-09-16T00:35:55.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:55 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f7eade8-dd2f-42ce-8b27-0c03f9172b79"}]': finished 2024-09-16T00:35:55.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:55 smithi031 ceph-mon[30956]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T00:35:55.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:35:55.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:55 smithi031 ceph-mon[30956]: from='client.? 172.21.15.81:0/980131825' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:35:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:55 smithi102 ceph-mon[35364]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:35:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:55 smithi102 ceph-mon[35364]: from='client.? 172.21.15.81:0/3255721000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f7eade8-dd2f-42ce-8b27-0c03f9172b79"}]: dispatch 2024-09-16T00:35:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:55 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f7eade8-dd2f-42ce-8b27-0c03f9172b79"}]: dispatch 2024-09-16T00:35:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:55 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f7eade8-dd2f-42ce-8b27-0c03f9172b79"}]': finished 2024-09-16T00:35:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:55 smithi102 ceph-mon[35364]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T00:35:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:35:55.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:55 smithi102 ceph-mon[35364]: from='client.? 172.21.15.81:0/980131825' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:35:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:57 smithi102 ceph-mon[35364]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:35:57.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:57 smithi031 ceph-mon[30956]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:35:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:57 smithi081 ceph-mon[35539]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:35:59.390 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:35:59 smithi081 ceph-mon[35539]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:35:59.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:35:59 smithi102 ceph-mon[35364]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:35:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:35:59 smithi031 ceph-mon[30956]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:01.356 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:01 smithi081 ceph-mon[35539]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:01.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:01 smithi102 ceph-mon[35364]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:01.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:01 smithi031 ceph-mon[30956]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:03.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:03 smithi081 ceph-mon[35539]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:03.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:03 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:36:03.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:03 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:03 smithi102 ceph-mon[35364]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:03 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:36:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:03 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:03.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:03 smithi031 ceph-mon[30956]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:03.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:03 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:36:03.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:03 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:04.406 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:04 smithi081 ceph-mon[35539]: Deploying daemon osd.7 on smithi081 2024-09-16T00:36:04.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:04 smithi102 ceph-mon[35364]: Deploying daemon osd.7 on smithi081 2024-09-16T00:36:04.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:04 smithi031 ceph-mon[30956]: Deploying daemon osd.7 on smithi081 2024-09-16T00:36:05.390 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:05 smithi081 ceph-mon[35539]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:05 smithi102 ceph-mon[35364]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:05.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:05 smithi031 ceph-mon[30956]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:07.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:07 smithi081 ceph-mon[35539]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:07.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:07 smithi102 ceph-mon[35364]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:07 smithi031 ceph-mon[30956]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:09.510 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:09 smithi081 ceph-mon[35539]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:09.510 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:09.510 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:09.510 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:09.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:09 smithi102 ceph-mon[35364]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:09.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:09.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:09.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:09.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:09 smithi031 ceph-mon[30956]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:11.426 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:11 smithi081 ceph-mon[35539]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:11.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:11 smithi102 ceph-mon[35364]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:11.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:11 smithi031 ceph-mon[30956]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:12.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:12.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:12.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:12.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:12.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:12 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:12.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:12.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:12 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:12.840 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 7 on host 'smithi081' 2024-09-16T00:36:12.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.839+0000 7efef3fff640 1 -- 172.21.15.81:0/2424726664 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efefc06a770 con 0x7efee80a3240 2024-09-16T00:36:12.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efee80a3240 msgr2=0x7efee80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:12.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efee80a3240 0x7efee80a5700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efee4000c00 tx=0x7efee40315d0 comp rx=0 tx=0).stop 2024-09-16T00:36:12.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc100ad0 msgr2=0x7efefc104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:12.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc100ad0 0x7efefc104a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efef400eaa0 tx=0x7efef400ef60 comp rx=0 tx=0).stop 2024-09-16T00:36:12.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 shutdown_connections 2024-09-16T00:36:12.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efee80a3240 0x7efee80a5700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:12.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efefc102690 0x7efefc1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:12.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efefc101cd0 0x7efefc1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:12.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 --2- 172.21.15.81:0/2424726664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efefc100ad0 0x7efefc104a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:12.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.841+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 >> 172.21.15.81:0/2424726664 conn(0x7efefc0fc280 msgr2=0x7efefc06a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:12.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.842+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 shutdown_connections 2024-09-16T00:36:12.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:36:12.842+0000 7eff01521640 1 -- 172.21.15.81:0/2424726664 wait complete. 2024-09-16T00:36:12.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:12.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:12.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:12.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:12.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:12 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:13.560 DEBUG:teuthology.orchestra.run.smithi081:osd.7> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T00:36:13.563 INFO:tasks.cephadm:Deploying osd.8 on smithi102 with /dev/vg_nvme/lv_2... 2024-09-16T00:36:13.563 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:36:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:13 smithi102 ceph-mon[35364]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:13 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:13 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:13 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:13.817 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:36:13.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:13 smithi081 ceph-mon[35539]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:13.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:13 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:13.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:13 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:13.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:13 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:13.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:13 smithi031 ceph-mon[30956]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:13.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:13 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:13.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:13 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:13 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:15.245 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:36:15 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:36:15.095+0000 7f311657b640 -1 osd.7 0 log_to_monitors true 2024-09-16T00:36:15.776 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:15 smithi081 ceph-mon[35539]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:15.776 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:15 smithi081 ceph-mon[35539]: from='osd.7 [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:36:15.776 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:15 smithi081 ceph-mon[35539]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:36:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:15 smithi102 ceph-mon[35364]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:15 smithi102 ceph-mon[35364]: from='osd.7 [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:36:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:15 smithi102 ceph-mon[35364]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:36:15.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:15 smithi031 ceph-mon[30956]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:15.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:15 smithi031 ceph-mon[30956]: from='osd.7 [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:36:15.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:15 smithi031 ceph-mon[30956]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:36:16.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:36:16.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='osd.7 [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:36:16.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:16.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:16 smithi102 ceph-mon[35364]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:36:16.586 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:36:16.614 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2024-09-16T00:36:16.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:36:16.511+0000 7f31124fe640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T00:36:16.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:36:16.521+0000 7f310e326640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:36:16.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:36:16.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='osd.7 [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:36:16.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:36:16.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:16.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:36:16.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:16.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:16.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:16.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:16.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:16.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:16.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:16.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:16.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:16 smithi081 ceph-mon[35539]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:36:16.865 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:36:16.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='osd.7 [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:16.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:16 smithi031 ceph-mon[30956]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:36:17.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.333+0000 7fe914e2a640 1 -- 172.21.15.102:0/4088834218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 msgr2=0x7fe910101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:17.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.333+0000 7fe914e2a640 1 --2- 172.21.15.102:0/4088834218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 0x7fe910101ff0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe900009900 tx=0x7fe90002a410 comp rx=0 tx=0).stop 2024-09-16T00:36:17.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.333+0000 7fe914e2a640 1 -- 172.21.15.102:0/4088834218 shutdown_connections 2024-09-16T00:36:17.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.333+0000 7fe914e2a640 1 --2- 172.21.15.102:0/4088834218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9101037b0 0x7fe91010a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:17.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.333+0000 7fe914e2a640 1 --2- 172.21.15.102:0/4088834218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe910102df0 0x7fe910103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:17.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.333+0000 7fe914e2a640 1 --2- 172.21.15.102:0/4088834218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 0x7fe910101ff0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:17.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.333+0000 7fe914e2a640 1 -- 172.21.15.102:0/4088834218 >> 172.21.15.102:0/4088834218 conn(0x7fe9100fd3c0 msgr2=0x7fe9100ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:17.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.333+0000 7fe914e2a640 1 -- 172.21.15.102:0/4088834218 shutdown_connections 2024-09-16T00:36:17.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.333+0000 7fe914e2a640 1 -- 172.21.15.102:0/4088834218 wait complete. 2024-09-16T00:36:17.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe914e2a640 1 Processor -- start 2024-09-16T00:36:17.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe914e2a640 1 -- start start 2024-09-16T00:36:17.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe914e2a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 0x7fe910197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:17.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe914e2a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe910102df0 0x7fe910198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:17.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe914e2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9101037b0 0x7fe91019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:17.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe914e2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010d8d0 con 0x7fe9101037b0 2024-09-16T00:36:17.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe914e2a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010d750 con 0x7fe910102df0 2024-09-16T00:36:17.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe914e2a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010da50 con 0x7fe910101bf0 2024-09-16T00:36:17.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe90e59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 0x7fe910197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:17.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe90e59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 0x7fe910197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60280/0 (socket says 172.21.15.102:60280) 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.334+0000 7fe90e59c640 1 -- 172.21.15.102:0/1393983502 learned_addr learned my addr 172.21.15.102:0/1393983502 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe907fff640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe910102df0 0x7fe910198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe90e59c640 1 -- 172.21.15.102:0/1393983502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe910102df0 msgr2=0x7fe910198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe90e59c640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe910102df0 0x7fe910198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe90e59c640 1 -- 172.21.15.102:0/1393983502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9101037b0 msgr2=0x7fe91019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe90e59c640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9101037b0 0x7fe91019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe90e59c640 1 -- 172.21.15.102:0/1393983502 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91019fb00 con 0x7fe910101bf0 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe90e59c640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 0x7fe910197ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe900031610 tx=0x7fe9000043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe9077fe640 1 -- 172.21.15.102:0/1393983502 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90003f030 con 0x7fe910101bf0 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91019fd90 con 0x7fe910101bf0 2024-09-16T00:36:17.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe9077fe640 1 -- 172.21.15.102:0/1393983502 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe900003ce0 con 0x7fe910101bf0 2024-09-16T00:36:17.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe9077fe640 1 -- 172.21.15.102:0/1393983502 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900035070 con 0x7fe910101bf0 2024-09-16T00:36:17.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.335+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9101a0230 con 0x7fe910101bf0 2024-09-16T00:36:17.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.337+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe910108fa0 con 0x7fe910101bf0 2024-09-16T00:36:17.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.337+0000 7fe9077fe640 1 -- 172.21.15.102:0/1393983502 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe900042030 con 0x7fe910101bf0 2024-09-16T00:36:17.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.338+0000 7fe9077fe640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe8fc0a3270 0x7fe8fc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:17.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.338+0000 7fe9077fe640 1 -- 172.21.15.102:0/1393983502 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7fe9000ef3f0 con 0x7fe910101bf0 2024-09-16T00:36:17.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.339+0000 7fe907fff640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe8fc0a3270 0x7fe8fc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:17.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.339+0000 7fe907fff640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe8fc0a3270 0x7fe8fc0a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9101993f0 tx=0x7fe8f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:17.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.342+0000 7fe9077fe640 1 -- 172.21.15.102:0/1393983502 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9000bcd90 con 0x7fe910101bf0 2024-09-16T00:36:17.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:17.526+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe910062630 con 0x7fe8fc0a3270 2024-09-16T00:36:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:17 smithi102 ceph-mon[35364]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:17 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi081 2024-09-16T00:36:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:17 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi081 to 5696M 2024-09-16T00:36:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:17 smithi102 ceph-mon[35364]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T00:36:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:17 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:17.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:17 smithi031 ceph-mon[30956]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:17.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:17 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi081 2024-09-16T00:36:17.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:17 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi081 to 5696M 2024-09-16T00:36:17.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:17 smithi031 ceph-mon[30956]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T00:36:17.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:17.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:17.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:17 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:17 smithi081 ceph-mon[35539]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:17 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi081 2024-09-16T00:36:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:17 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi081 to 5696M 2024-09-16T00:36:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:17 smithi081 ceph-mon[35539]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T00:36:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:17 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:18.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:18 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:36:18.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:18 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:36:18.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:18 smithi102 ceph-mon[35364]: osd.7 [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] boot 2024-09-16T00:36:18.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:18 smithi102 ceph-mon[35364]: osdmap e49: 8 total, 8 up, 8 in 2024-09-16T00:36:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:18 smithi102 ceph-mon[35364]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:36:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:36:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:18.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:18 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:36:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:18 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:36:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:18 smithi031 ceph-mon[30956]: osd.7 [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] boot 2024-09-16T00:36:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:18 smithi031 ceph-mon[30956]: osdmap e49: 8 total, 8 up, 8 in 2024-09-16T00:36:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:18 smithi031 ceph-mon[30956]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:36:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:36:18.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:18 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:36:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:18 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:36:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:18 smithi081 ceph-mon[35539]: osd.7 [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] boot 2024-09-16T00:36:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:18 smithi081 ceph-mon[35539]: osdmap e49: 8 total, 8 up, 8 in 2024-09-16T00:36:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:36:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:18 smithi081 ceph-mon[35539]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:36:18.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:36:18.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:19.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:19 smithi102 ceph-mon[35364]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:19.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:19 smithi102 ceph-mon[35364]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:36:19.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:19 smithi102 ceph-mon[35364]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T00:36:19.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:19 smithi031 ceph-mon[30956]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:19.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:19 smithi031 ceph-mon[30956]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:36:19.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:19 smithi031 ceph-mon[30956]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T00:36:19.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:19 smithi081 ceph-mon[35539]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:36:19.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:19 smithi081 ceph-mon[35539]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:36:19.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:19 smithi081 ceph-mon[35539]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T00:36:20.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:20 smithi102 ceph-mon[35364]: osdmap e52: 8 total, 8 up, 8 in 2024-09-16T00:36:20.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:20 smithi102 ceph-mon[35364]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:20.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:20 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/1784656245' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2673057-c859-4b46-92af-c376f007503a"}]: dispatch 2024-09-16T00:36:20.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:20 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2673057-c859-4b46-92af-c376f007503a"}]: dispatch 2024-09-16T00:36:20.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:20 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2673057-c859-4b46-92af-c376f007503a"}]': finished 2024-09-16T00:36:20.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:20 smithi102 ceph-mon[35364]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:36:20.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:20.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:20 smithi031 ceph-mon[30956]: osdmap e52: 8 total, 8 up, 8 in 2024-09-16T00:36:20.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:20 smithi031 ceph-mon[30956]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:20.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:20 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/1784656245' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2673057-c859-4b46-92af-c376f007503a"}]: dispatch 2024-09-16T00:36:20.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:20 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2673057-c859-4b46-92af-c376f007503a"}]: dispatch 2024-09-16T00:36:20.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:20 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2673057-c859-4b46-92af-c376f007503a"}]': finished 2024-09-16T00:36:20.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:20 smithi031 ceph-mon[30956]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:36:20.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:20.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:20 smithi081 ceph-mon[35539]: osdmap e52: 8 total, 8 up, 8 in 2024-09-16T00:36:20.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:20 smithi081 ceph-mon[35539]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:20.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:20 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/1784656245' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2673057-c859-4b46-92af-c376f007503a"}]: dispatch 2024-09-16T00:36:20.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:20 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2673057-c859-4b46-92af-c376f007503a"}]: dispatch 2024-09-16T00:36:20.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:20 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2673057-c859-4b46-92af-c376f007503a"}]': finished 2024-09-16T00:36:20.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:20 smithi081 ceph-mon[35539]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:36:20.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:21.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:21 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/899411729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:36:21.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:21 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/899411729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:36:22.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:21 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/899411729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:36:22.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:22 smithi031 ceph-mon[30956]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:22.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:22 smithi081 ceph-mon[35539]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:23.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:22 smithi102 ceph-mon[35364]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:24.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:24 smithi081 ceph-mon[35539]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:25.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:24 smithi102 ceph-mon[35364]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:25.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:24 smithi031 ceph-mon[30956]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:26.875 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:26 smithi102 ceph-mon[35364]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:26.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:26 smithi031 ceph-mon[30956]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:26.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:26 smithi081 ceph-mon[35539]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:28.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:28 smithi081 ceph-mon[35539]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:28.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:28 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:36:28.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:28 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:28.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:28 smithi081 ceph-mon[35539]: Deploying daemon osd.8 on smithi102 2024-09-16T00:36:29.012 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:28 smithi102 ceph-mon[35364]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:29.012 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:28 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:36:29.012 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:28 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:29.012 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:28 smithi102 ceph-mon[35364]: Deploying daemon osd.8 on smithi102 2024-09-16T00:36:29.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:28 smithi031 ceph-mon[30956]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:29.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:28 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:36:29.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:28 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:29.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:28 smithi031 ceph-mon[30956]: Deploying daemon osd.8 on smithi102 2024-09-16T00:36:30.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:30 smithi081 ceph-mon[35539]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:31.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:30 smithi102 ceph-mon[35364]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:31.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:30 smithi031 ceph-mon[30956]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:32.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:32 smithi081 ceph-mon[35539]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:33.028 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:32 smithi102 ceph-mon[35364]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:33.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:32 smithi031 ceph-mon[30956]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:34.949 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:34 smithi102 ceph-mon[35364]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:34.949 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:34.949 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:34.949 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:34.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:34 smithi081 ceph-mon[35539]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:34.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:34.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:34.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:35.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:34 smithi031 ceph-mon[30956]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:35.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:35.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:35.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:36.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:36 smithi081 ceph-mon[35539]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:37.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:36 smithi102 ceph-mon[35364]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:36 smithi031 ceph-mon[30956]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:37.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.924+0000 7fe9077fe640 1 -- 172.21.15.102:0/1393983502 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe910062630 con 0x7fe8fc0a3270 2024-09-16T00:36:37.925 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 8 on host 'smithi102' 2024-09-16T00:36:37.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe8fc0a3270 msgr2=0x7fe8fc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:37.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe8fc0a3270 0x7fe8fc0a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9101993f0 tx=0x7fe8f8008040 comp rx=0 tx=0).stop 2024-09-16T00:36:37.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 msgr2=0x7fe910197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:37.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 0x7fe910197ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe900031610 tx=0x7fe9000043b0 comp rx=0 tx=0).stop 2024-09-16T00:36:37.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 shutdown_connections 2024-09-16T00:36:37.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9101037b0 0x7fe91019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:37.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe910102df0 0x7fe910198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:37.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe8fc0a3270 0x7fe8fc0a5730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:37.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 --2- 172.21.15.102:0/1393983502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe910101bf0 0x7fe910197ed0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:37.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 >> 172.21.15.102:0/1393983502 conn(0x7fe9100fd3c0 msgr2=0x7fe910106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:37.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 shutdown_connections 2024-09-16T00:36:37.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:36:37.927+0000 7fe914e2a640 1 -- 172.21.15.102:0/1393983502 wait complete. 2024-09-16T00:36:38.470 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.471 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.471 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:38.471 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:38.471 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.471 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:38.471 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.471 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.607 DEBUG:teuthology.orchestra.run.smithi102:osd.8> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.8.service 2024-09-16T00:36:38.610 INFO:tasks.cephadm:Deploying osd.9 on smithi031 with /dev/vg_nvme/lv_1... 2024-09-16T00:36:38.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:36:38.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:38.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:38.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:36:38.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:38.879 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:39.329 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:39 smithi031 ceph-mon[30956]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:39.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:39 smithi081 ceph-mon[35539]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:39.505 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:39 smithi102 ceph-mon[35364]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:40.335 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:40 smithi102 ceph-mon[35364]: from='osd.8 [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:36:40.335 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:40 smithi102 ceph-mon[35364]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:36:40.335 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:36:40 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:36:40.065+0000 7fb47115d640 -1 osd.8 0 log_to_monitors true 2024-09-16T00:36:40.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:40 smithi031 ceph-mon[30956]: from='osd.8 [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:36:40.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:40 smithi031 ceph-mon[30956]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:36:40.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:40 smithi081 ceph-mon[35539]: from='osd.8 [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:36:40.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:40 smithi081 ceph-mon[35539]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:36:41.464 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:36:41 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:36:41.224+0000 7fb46d0e0640 -1 osd.8 0 waiting for initial osdmap 2024-09-16T00:36:41.464 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:36:41 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:36:41.233+0000 7fb468f08640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:36:41.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:41 smithi102 ceph-mon[35364]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:41.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:41 smithi102 ceph-mon[35364]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:36:41.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:41 smithi102 ceph-mon[35364]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:36:41.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:41.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:41 smithi102 ceph-mon[35364]: from='osd.8 [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:36:41.465 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:41 smithi102 ceph-mon[35364]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:36:41.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:41 smithi031 ceph-mon[30956]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:41.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:41 smithi031 ceph-mon[30956]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:36:41.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:41 smithi031 ceph-mon[30956]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:36:41.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:41.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:41 smithi031 ceph-mon[30956]: from='osd.8 [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:36:41.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:41 smithi031 ceph-mon[30956]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:36:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:36:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:41 smithi081 ceph-mon[35539]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:41 smithi081 ceph-mon[35539]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:36:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:41 smithi081 ceph-mon[35539]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:36:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:41 smithi081 ceph-mon[35539]: from='osd.8 [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:36:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:41 smithi081 ceph-mon[35539]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:36:41.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_1 2024-09-16T00:36:41.819 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:36:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.270+0000 7f0636eee640 1 -- 172.21.15.31:0/1317798923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301037c0 msgr2=0x7f063010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.270+0000 7f0636eee640 1 --2- 172.21.15.31:0/1317798923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301037c0 0x7f063010a050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0628009ca0 tx=0x7f062802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:36:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.271+0000 7f0636eee640 1 -- 172.21.15.31:0/1317798923 shutdown_connections 2024-09-16T00:36:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.271+0000 7f0636eee640 1 --2- 172.21.15.31:0/1317798923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301037c0 0x7f063010a050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.271+0000 7f0636eee640 1 --2- 172.21.15.31:0/1317798923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0630102e00 0x7f0630103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.271+0000 7f0636eee640 1 --2- 172.21.15.31:0/1317798923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0630101c00 0x7f0630102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.271+0000 7f0636eee640 1 -- 172.21.15.31:0/1317798923 >> 172.21.15.31:0/1317798923 conn(0x7f06300fd390 msgr2=0x7f06300ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.271+0000 7f0636eee640 1 -- 172.21.15.31:0/1317798923 shutdown_connections 2024-09-16T00:36:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.271+0000 7f0636eee640 1 -- 172.21.15.31:0/1317798923 wait complete. 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.271+0000 7f0636eee640 1 Processor -- start 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.271+0000 7f0636eee640 1 -- start start 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0636eee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0630101c00 0x7f063019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0636eee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0630102e00 0x7f063019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0636eee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301037c0 0x7f06301a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0636eee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0630076020 con 0x7f06301037c0 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0636eee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0630075ea0 con 0x7f0630102e00 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0636eee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06300761a0 con 0x7f0630101c00 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0634c8a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0630101c00 0x7f063019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0634c8a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0630101c00 0x7f063019e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:46050/0 (socket says 172.21.15.31:46050) 2024-09-16T00:36:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0634c8a640 1 -- 172.21.15.31:0/2304551032 learned_addr learned my addr 172.21.15.31:0/2304551032 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f063548b640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301037c0 0x7f06301a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f062ffff640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0630102e00 0x7f063019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0634c8a640 1 -- 172.21.15.31:0/2304551032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0630102e00 msgr2=0x7f063019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0634c8a640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0630102e00 0x7f063019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0634c8a640 1 -- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301037c0 msgr2=0x7f06301a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0634c8a640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301037c0 0x7f06301a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0634c8a640 1 -- 172.21.15.31:0/2304551032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06301a60c0 con 0x7f0630101c00 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.272+0000 7f0634c8a640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0630101c00 0x7f063019e490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f062400d770 tx=0x7f062400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.273+0000 7f062dffb640 1 -- 172.21.15.31:0/2304551032 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624017070 con 0x7f0630101c00 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.273+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06301a63b0 con 0x7f0630101c00 2024-09-16T00:36:42.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.273+0000 7f062dffb640 1 -- 172.21.15.31:0/2304551032 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0624009c30 con 0x7f0630101c00 2024-09-16T00:36:42.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.273+0000 7f062dffb640 1 -- 172.21.15.31:0/2304551032 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624003dd0 con 0x7f0630101c00 2024-09-16T00:36:42.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.273+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06301a68f0 con 0x7f0630101c00 2024-09-16T00:36:42.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.274+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0630105730 con 0x7f0630101c00 2024-09-16T00:36:42.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.275+0000 7f062dffb640 1 -- 172.21.15.31:0/2304551032 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0624003f70 con 0x7f0630101c00 2024-09-16T00:36:42.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.275+0000 7f062dffb640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f06200a3270 0x7f06200a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:42.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.276+0000 7f062dffb640 1 -- 172.21.15.31:0/2304551032 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5862+0+0 (secure 0 0 0) 0x7f06240ce830 con 0x7f0630101c00 2024-09-16T00:36:42.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.276+0000 7f062ffff640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f06200a3270 0x7f06200a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:42.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.276+0000 7f062ffff640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f06200a3270 0x7f06200a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f061c0095f0 tx=0x7f061c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:42.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.280+0000 7f062dffb640 1 -- 172.21.15.31:0/2304551032 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0624009de0 con 0x7f0630101c00 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: osdmap e55: 9 total, 8 up, 9 in 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi102 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi102 to 5696M 2024-09-16T00:36:42.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:42.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:42.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:36:42.464+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0630062630 con 0x7f06200a3270 2024-09-16T00:36:42.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:36:42.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: osdmap e55: 9 total, 8 up, 9 in 2024-09-16T00:36:42.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:42.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi102 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi102 to 5696M 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:42.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: osdmap e55: 9 total, 8 up, 9 in 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi102 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:36:42.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi102 to 5696M 2024-09-16T00:36:42.551 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:42.551 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:42.551 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:36:42.551 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:36:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:36:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:36:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: osd.8 [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] boot 2024-09-16T00:36:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T00:36:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:43.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:43.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:36:43.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:36:43.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:43 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:36:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:36:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:43.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:43.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: osd.8 [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] boot 2024-09-16T00:36:43.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T00:36:43.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:43.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:43.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:36:43.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:36:43.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:43 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:43.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: osd.8 [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] boot 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:36:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:43 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:44 smithi081 ceph-mon[35539]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:36:44.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:44 smithi031 ceph-mon[30956]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:36:44.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:44 smithi102 ceph-mon[35364]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:36:45.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:45 smithi081 ceph-mon[35539]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:36:45.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:45 smithi081 ceph-mon[35539]: osdmap e58: 9 total, 9 up, 9 in 2024-09-16T00:36:45.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:45 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1523983938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf12fcdc-621d-44c2-8863-e995d82fb5a7"}]: dispatch 2024-09-16T00:36:45.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:45 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1523983938' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf12fcdc-621d-44c2-8863-e995d82fb5a7"}]': finished 2024-09-16T00:36:45.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:45 smithi081 ceph-mon[35539]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T00:36:45.499 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:36:45.563 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:45 smithi102 ceph-mon[35364]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:36:45.563 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:45 smithi102 ceph-mon[35364]: osdmap e58: 9 total, 9 up, 9 in 2024-09-16T00:36:45.563 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:45 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1523983938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf12fcdc-621d-44c2-8863-e995d82fb5a7"}]: dispatch 2024-09-16T00:36:45.563 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:45 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1523983938' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf12fcdc-621d-44c2-8863-e995d82fb5a7"}]': finished 2024-09-16T00:36:45.563 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:45 smithi102 ceph-mon[35364]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T00:36:45.564 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:36:45.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:45 smithi031 ceph-mon[30956]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:36:45.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:45 smithi031 ceph-mon[30956]: osdmap e58: 9 total, 9 up, 9 in 2024-09-16T00:36:45.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:45 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1523983938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf12fcdc-621d-44c2-8863-e995d82fb5a7"}]: dispatch 2024-09-16T00:36:45.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:45 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1523983938' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf12fcdc-621d-44c2-8863-e995d82fb5a7"}]': finished 2024-09-16T00:36:45.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:45 smithi031 ceph-mon[30956]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T00:36:45.620 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:36:46.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:46 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/804432326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:36:46.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:46 smithi081 ceph-mon[35539]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:36:46.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:36:46.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:46 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/804432326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:36:46.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:46 smithi102 ceph-mon[35364]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:36:46.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:36:46.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:46 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/804432326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:36:46.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:46 smithi031 ceph-mon[30956]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:36:46.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:36:47.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:47 smithi081 ceph-mon[35539]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:36:47.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:47 smithi102 ceph-mon[35364]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:36:47.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:47 smithi031 ceph-mon[30956]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:36:49.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:49 smithi102 ceph-mon[35364]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:36:49.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:49 smithi031 ceph-mon[30956]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:36:49.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:49 smithi081 ceph-mon[35539]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:36:51.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:51 smithi102 ceph-mon[35364]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:36:51.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:51 smithi031 ceph-mon[30956]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:36:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:51 smithi081 ceph-mon[35539]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:36:53.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:53 smithi102 ceph-mon[35364]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-16T00:36:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:53 smithi031 ceph-mon[30956]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-16T00:36:53.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:53 smithi081 ceph-mon[35539]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-16T00:36:54.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:54 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:36:54.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:54 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:54.581 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:54 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:36:54.581 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:54 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:54.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:54 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:36:54.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:54 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:36:55.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:55 smithi031 ceph-mon[30956]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-16T00:36:55.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:55 smithi031 ceph-mon[30956]: Deploying daemon osd.9 on smithi031 2024-09-16T00:36:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:55 smithi102 ceph-mon[35364]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-16T00:36:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:55 smithi102 ceph-mon[35364]: Deploying daemon osd.9 on smithi031 2024-09-16T00:36:55.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:55 smithi081 ceph-mon[35539]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-16T00:36:55.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:55 smithi081 ceph-mon[35539]: Deploying daemon osd.9 on smithi031 2024-09-16T00:36:57.534 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:57 smithi031 ceph-mon[30956]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:36:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:57 smithi102 ceph-mon[35364]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:36:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:57 smithi081 ceph-mon[35539]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:36:59.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:36:59 smithi031 ceph-mon[30956]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T00:36:59.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:36:59 smithi102 ceph-mon[35364]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T00:36:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:36:59 smithi081 ceph-mon[35539]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T00:37:00.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:00 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:00.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:00 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:00.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:00 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:00.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:00 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:00.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:00 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:00.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:00 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:00 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:00 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:00 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:01.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:01 smithi102 ceph-mon[35364]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:37:01.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:01 smithi031 ceph-mon[30956]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:37:01.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:01 smithi081 ceph-mon[35539]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:37:03.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:03 smithi031 ceph-mon[30956]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:03.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:03 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:03.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:03 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:03.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:03 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:03.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:03 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:03.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:03 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:03 smithi102 ceph-mon[35364]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:03.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:03 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:03.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:03 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:03.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:03 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:03.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:03 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:03.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:03 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:03.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:03 smithi081 ceph-mon[35539]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:03.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:03 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:03.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:03 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:03.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:03 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:03.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:03 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:03.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:03 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:03.883 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 9 on host 'smithi031' 2024-09-16T00:37:03.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.882+0000 7f062dffb640 1 -- 172.21.15.31:0/2304551032 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0630062630 con 0x7f06200a3270 2024-09-16T00:37:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.884+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f06200a3270 msgr2=0x7f06200a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.884+0000 7f0636eee640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f06200a3270 0x7f06200a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f061c0095f0 tx=0x7f061c005f90 comp rx=0 tx=0).stop 2024-09-16T00:37:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0630101c00 msgr2=0x7f063019e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0630101c00 0x7f063019e490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f062400d770 tx=0x7f062400dc30 comp rx=0 tx=0).stop 2024-09-16T00:37:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 shutdown_connections 2024-09-16T00:37:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301037c0 0x7f06301a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0630102e00 0x7f063019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f06200a3270 0x7f06200a5730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 --2- 172.21.15.31:0/2304551032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0630101c00 0x7f063019e490 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 >> 172.21.15.31:0/2304551032 conn(0x7f06300fd390 msgr2=0x7f063010e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 shutdown_connections 2024-09-16T00:37:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:03.885+0000 7f0636eee640 1 -- 172.21.15.31:0/2304551032 wait complete. 2024-09-16T00:37:04.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:04.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:04.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:04 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:04.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:04.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:04.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:04 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:04.704 DEBUG:teuthology.orchestra.run.smithi031:osd.9> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.9.service 2024-09-16T00:37:04.706 INFO:tasks.cephadm:Deploying osd.10 on smithi081 with /dev/vg_nvme/lv_1... 2024-09-16T00:37:04.706 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:37:04.734 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:04.734 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:04.735 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:04 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:04.961 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:37:05.365 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:05 smithi081 ceph-mon[35539]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:05 smithi102 ceph-mon[35364]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:05.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:05 smithi031 ceph-mon[30956]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:06.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:37:05 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:37:05.998+0000 7f3f365f8640 -1 osd.9 0 log_to_monitors true 2024-09-16T00:37:06.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:06 smithi102 ceph-mon[35364]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:37:06.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:06 smithi031 ceph-mon[30956]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:37:06.646 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:06 smithi081 ceph-mon[35539]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:37:07.612 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:07 smithi081 ceph-mon[35539]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:07.612 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:07 smithi081 ceph-mon[35539]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:37:07.612 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:07 smithi081 ceph-mon[35539]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:37:07.612 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:07 smithi081 ceph-mon[35539]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:37:07.612 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:07 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:07 smithi031 ceph-mon[30956]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:07 smithi031 ceph-mon[30956]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:37:07.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:07 smithi031 ceph-mon[30956]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:37:07.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:07 smithi031 ceph-mon[30956]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:37:07.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:07 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:07.785 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-16T00:37:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:07 smithi102 ceph-mon[35364]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:07 smithi102 ceph-mon[35364]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:37:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:07 smithi102 ceph-mon[35364]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:37:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:07 smithi102 ceph-mon[35364]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T00:37:07.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:07 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:07.812 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_1 2024-09-16T00:37:08.105 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:37:08.352 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:37:07 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:37:07.991+0000 7f3f3257b640 -1 osd.9 0 waiting for initial osdmap 2024-09-16T00:37:08.352 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:37:08.005+0000 7f3f2a39b640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:37:08.477 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:37:08.477 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: osdmap e62: 10 total, 9 up, 10 in 2024-09-16T00:37:08.477 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' 2024-09-16T00:37:08.478 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:08.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 -- 172.21.15.81:0/869084438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 msgr2=0x7f2b8c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/869084438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 0x7f2b8c100f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009900 tx=0x7f2b7c02a410 comp rx=0 tx=0).stop 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 -- 172.21.15.81:0/869084438 shutdown_connections 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/869084438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b8c1026c0 0x7f2b8c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/869084438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c101d00 0x7f2b8c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/869084438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 0x7f2b8c100f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 -- 172.21.15.81:0/869084438 >> 172.21.15.81:0/869084438 conn(0x7f2b8c0fc290 msgr2=0x7f2b8c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 -- 172.21.15.81:0/869084438 shutdown_connections 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 -- 172.21.15.81:0/869084438 wait complete. 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 Processor -- start 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.560+0000 7f2b91cb9640 1 -- start start 2024-09-16T00:37:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b91cb9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 0x7f2b8c196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b91cb9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b8c101d00 0x7f2b8c1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b91cb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c1026c0 0x7f2b8c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b91cb9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c10c7e0 con 0x7f2b8c1026c0 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b91cb9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c10c660 con 0x7f2b8c100b00 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b91cb9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c10c960 con 0x7f2b8c101d00 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 0x7f2b8c196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 0x7f2b8c196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56690/0 (socket says 172.21.15.81:56690) 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8b7fe640 1 -- 172.21.15.81:0/2745490715 learned_addr learned my addr 172.21.15.81:0/2745490715 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8bfff640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c1026c0 0x7f2b8c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b83fff640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b8c101d00 0x7f2b8c1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8b7fe640 1 -- 172.21.15.81:0/2745490715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b8c101d00 msgr2=0x7f2b8c1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8b7fe640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b8c101d00 0x7f2b8c1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8b7fe640 1 -- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c1026c0 msgr2=0x7f2b8c19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8b7fe640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c1026c0 0x7f2b8c19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8b7fe640 1 -- 172.21.15.81:0/2745490715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c19eaa0 con 0x7f2b8c100b00 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b8b7fe640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 0x7f2b8c196e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c031610 tx=0x7f2b7c0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b897fa640 1 -- 172.21.15.81:0/2745490715 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c03f030 con 0x7f2b8c100b00 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b897fa640 1 -- 172.21.15.81:0/2745490715 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b7c003ce0 con 0x7f2b8c100b00 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b897fa640 1 -- 172.21.15.81:0/2745490715 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c035070 con 0x7f2b8c100b00 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.561+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c19ed30 con 0x7f2b8c100b00 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.562+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c19f230 con 0x7f2b8c100b00 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.563+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c107ef0 con 0x7f2b8c100b00 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.564+0000 7f2b897fa640 1 -- 172.21.15.81:0/2745490715 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2b7c042030 con 0x7f2b8c100b00 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.564+0000 7f2b897fa640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b780a32f0 0x7f2b780a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.564+0000 7f2b897fa640 1 -- 172.21.15.81:0/2745490715 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7f2b7c0ef760 con 0x7f2b8c100b00 2024-09-16T00:37:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.565+0000 7f2b83fff640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b780a32f0 0x7f2b780a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.565+0000 7f2b83fff640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b780a32f0 0x7f2b780a57b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b74005fd0 tx=0x7f2b74005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.568+0000 7f2b897fa640 1 -- 172.21.15.81:0/2745490715 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b7c0bcda0 con 0x7f2b8c100b00 2024-09-16T00:37:08.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: osdmap e62: 10 total, 9 up, 10 in 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:08.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:08.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' 2024-09-16T00:37:08.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:08.742 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:08.741+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2b8c062630 con 0x7f2b780a32f0 2024-09-16T00:37:08.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: osdmap e62: 10 total, 9 up, 10 in 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:08.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:08.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:08.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894]' entity='osd.9' 2024-09-16T00:37:08.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:09.463 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi031 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] boot 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: osdmap e63: 10 total, 10 up, 10 in 2024-09-16T00:37:09.464 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:09 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:09.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi031 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] boot 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: osdmap e63: 10 total, 10 up, 10 in 2024-09-16T00:37:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:09 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:09.701 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:37:09.701 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:37:09.701 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:37:09.701 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi031 2024-09-16T00:37:09.701 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-16T00:37:09.701 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:37:09.701 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:37:09.702 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:09.702 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: osd.9 [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] boot 2024-09-16T00:37:09.702 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: osdmap e63: 10 total, 10 up, 10 in 2024-09-16T00:37:09.702 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:09 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:37:10.487 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:10 smithi081 ceph-mon[35539]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:10.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:10 smithi031 ceph-mon[30956]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:10.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:10 smithi102 ceph-mon[35364]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:11.612 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:11 smithi081 ceph-mon[35539]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:11.612 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:11 smithi081 ceph-mon[35539]: osdmap e64: 10 total, 10 up, 10 in 2024-09-16T00:37:11.612 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:11 smithi081 ceph-mon[35539]: from='client.? 172.21.15.81:0/3331257266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d229e783-ab63-44ce-816e-bf0a1949c5cc"}]: dispatch 2024-09-16T00:37:11.613 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:11 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d229e783-ab63-44ce-816e-bf0a1949c5cc"}]: dispatch 2024-09-16T00:37:11.613 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:11 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d229e783-ab63-44ce-816e-bf0a1949c5cc"}]': finished 2024-09-16T00:37:11.613 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:11 smithi081 ceph-mon[35539]: osdmap e65: 11 total, 10 up, 11 in 2024-09-16T00:37:11.613 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:11.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:11 smithi031 ceph-mon[30956]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:11.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:11 smithi031 ceph-mon[30956]: osdmap e64: 10 total, 10 up, 10 in 2024-09-16T00:37:11.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:11 smithi031 ceph-mon[30956]: from='client.? 172.21.15.81:0/3331257266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d229e783-ab63-44ce-816e-bf0a1949c5cc"}]: dispatch 2024-09-16T00:37:11.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:11 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d229e783-ab63-44ce-816e-bf0a1949c5cc"}]: dispatch 2024-09-16T00:37:11.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:11 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d229e783-ab63-44ce-816e-bf0a1949c5cc"}]': finished 2024-09-16T00:37:11.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:11 smithi031 ceph-mon[30956]: osdmap e65: 11 total, 10 up, 11 in 2024-09-16T00:37:11.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:11.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:11 smithi102 ceph-mon[35364]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:11.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:11 smithi102 ceph-mon[35364]: osdmap e64: 10 total, 10 up, 10 in 2024-09-16T00:37:11.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:11 smithi102 ceph-mon[35364]: from='client.? 172.21.15.81:0/3331257266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d229e783-ab63-44ce-816e-bf0a1949c5cc"}]: dispatch 2024-09-16T00:37:11.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:11 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d229e783-ab63-44ce-816e-bf0a1949c5cc"}]: dispatch 2024-09-16T00:37:11.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:11 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d229e783-ab63-44ce-816e-bf0a1949c5cc"}]': finished 2024-09-16T00:37:11.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:11 smithi102 ceph-mon[35364]: osdmap e65: 11 total, 10 up, 11 in 2024-09-16T00:37:11.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:12.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:12 smithi031 ceph-mon[30956]: from='client.? 172.21.15.81:0/1650079513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:37:12.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:12 smithi081 ceph-mon[35539]: from='client.? 172.21.15.81:0/1650079513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:37:12.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:12 smithi102 ceph-mon[35364]: from='client.? 172.21.15.81:0/1650079513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:37:13.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:13 smithi031 ceph-mon[30956]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:13.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:13 smithi081 ceph-mon[35539]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:13 smithi102 ceph-mon[35364]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:15.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:15 smithi031 ceph-mon[30956]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:15.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:15 smithi081 ceph-mon[35539]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:15 smithi102 ceph-mon[35364]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:17.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:17 smithi031 ceph-mon[30956]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:17.626 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:17 smithi081 ceph-mon[35539]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:17 smithi102 ceph-mon[35364]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:19.648 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:19 smithi081 ceph-mon[35539]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:19 smithi102 ceph-mon[35364]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:19.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:19 smithi031 ceph-mon[30956]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:20.589 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:37:20.589 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:20 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:20.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:37:20.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:20 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:20.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:37:20.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:20 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:21.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:21 smithi081 ceph-mon[35539]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:21.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:21 smithi081 ceph-mon[35539]: Deploying daemon osd.10 on smithi081 2024-09-16T00:37:21.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:21 smithi102 ceph-mon[35364]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:21.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:21 smithi102 ceph-mon[35364]: Deploying daemon osd.10 on smithi081 2024-09-16T00:37:21.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:21 smithi031 ceph-mon[30956]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:21.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:21 smithi031 ceph-mon[30956]: Deploying daemon osd.10 on smithi081 2024-09-16T00:37:23.473 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:23 smithi081 ceph-mon[35539]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:23.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:23 smithi102 ceph-mon[35364]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:23.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:23 smithi031 ceph-mon[30956]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:25.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:25 smithi081 ceph-mon[35539]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:25.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:25 smithi102 ceph-mon[35364]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:25.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:25 smithi031 ceph-mon[30956]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:26.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:26.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:26.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:26.778 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:26.778 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:26.778 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:26.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:26.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:26.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:27.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:27 smithi081 ceph-mon[35539]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:27.795 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:27 smithi031 ceph-mon[30956]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:27.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:27 smithi102 ceph-mon[35364]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:29.156 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 10 on host 'smithi081' 2024-09-16T00:37:29.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.155+0000 7f2b897fa640 1 -- 172.21.15.81:0/2745490715 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2b8c062630 con 0x7f2b780a32f0 2024-09-16T00:37:29.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b780a32f0 msgr2=0x7f2b780a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:29.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b780a32f0 0x7f2b780a57b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b74005fd0 tx=0x7f2b74005850 comp rx=0 tx=0).stop 2024-09-16T00:37:29.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 msgr2=0x7f2b8c196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:29.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 0x7f2b8c196e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c031610 tx=0x7f2b7c0043b0 comp rx=0 tx=0).stop 2024-09-16T00:37:29.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 shutdown_connections 2024-09-16T00:37:29.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c1026c0 0x7f2b8c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:29.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b780a32f0 0x7f2b780a57b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:29.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b8c101d00 0x7f2b8c1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:29.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 --2- 172.21.15.81:0/2745490715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b8c100b00 0x7f2b8c196e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:29.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 >> 172.21.15.81:0/2745490715 conn(0x7f2b8c0fc290 msgr2=0x7f2b8c1050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:29.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 shutdown_connections 2024-09-16T00:37:29.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-16T00:37:29.158+0000 7f2b91cb9640 1 -- 172.21.15.81:0/2745490715 wait complete. 2024-09-16T00:37:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:29 smithi081 ceph-mon[35539]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:29.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:29 smithi102 ceph-mon[35364]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.865 DEBUG:teuthology.orchestra.run.smithi081:osd.10> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.10.service 2024-09-16T00:37:29.868 INFO:tasks.cephadm:Deploying osd.11 on smithi102 with /dev/vg_nvme/lv_1... 2024-09-16T00:37:29.868 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:37:29.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:29 smithi031 ceph-mon[30956]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:29.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:29.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:29.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:29.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:29.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:30.120 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:37:31.505 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:31 smithi102 ceph-mon[35364]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:31.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:31 smithi081 ceph-mon[35539]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:31.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:37:31 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:37:31.418+0000 7fc51d8f7640 -1 osd.10 0 log_to_monitors true 2024-09-16T00:37:31.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:31 smithi031 ceph-mon[30956]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:32.489 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:32 smithi102 ceph-mon[35364]: from='osd.10 [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:37:32.489 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:32 smithi102 ceph-mon[35364]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:37:32.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:32 smithi081 ceph-mon[35539]: from='osd.10 [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:37:32.639 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:32 smithi081 ceph-mon[35539]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:37:32.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:32 smithi031 ceph-mon[30956]: from='osd.10 [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:37:32.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:32 smithi031 ceph-mon[30956]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:37:33.059 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T00:37:33.085 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_1 2024-09-16T00:37:33.335 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:37:33.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:33.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:37:33.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='osd.10 [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:37:33.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:37:33.727 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:33.728 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:33 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:37:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='osd.10 [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:37:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:37:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:37:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:33.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:33.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:33 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.809+0000 7f1e2cc62640 1 -- 172.21.15.102:0/864825264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 msgr2=0x7f1e28100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:33.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.809+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/864825264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 0x7f1e28100ed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009900 tx=0x7f1e1802a410 comp rx=0 tx=0).stop 2024-09-16T00:37:33.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.809+0000 7f1e2cc62640 1 -- 172.21.15.102:0/864825264 shutdown_connections 2024-09-16T00:37:33.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.809+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/864825264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e28102690 0x7f1e28108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:33.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.809+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/864825264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e28101cd0 0x7f1e28102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.809+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/864825264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 0x7f1e28100ed0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.809+0000 7f1e2cc62640 1 -- 172.21.15.102:0/864825264 >> 172.21.15.102:0/864825264 conn(0x7f1e280fc280 msgr2=0x7f1e280fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.809+0000 7f1e2cc62640 1 -- 172.21.15.102:0/864825264 shutdown_connections 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.809+0000 7f1e2cc62640 1 -- 172.21.15.102:0/864825264 wait complete. 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2cc62640 1 Processor -- start 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2cc62640 1 -- start start 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2cc62640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 0x7f1e2810b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2cc62640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e28101cd0 0x7f1e28109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2cc62640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e28102690 0x7f1e28109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2cc62640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2810d880 con 0x7f1e28101cd0 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2cc62640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2810d700 con 0x7f1e28102690 2024-09-16T00:37:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2cc62640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2810da00 con 0x7f1e28100ad0 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 0x7f1e2810b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 0x7f1e2810b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43562/0 (socket says 172.21.15.102:43562) 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.810+0000 7f1e2659c640 1 -- 172.21.15.102:0/4108232507 learned_addr learned my addr 172.21.15.102:0/4108232507 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e26d9d640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e28102690 0x7f1e28109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e1ffff640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e28101cd0 0x7f1e28109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e2659c640 1 -- 172.21.15.102:0/4108232507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e28102690 msgr2=0x7f1e28109fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e2659c640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e28102690 0x7f1e28109fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e2659c640 1 -- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e28101cd0 msgr2=0x7f1e28109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e2659c640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e28101cd0 0x7f1e28109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e2659c640 1 -- 172.21.15.102:0/4108232507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2810a8a0 con 0x7f1e28100ad0 2024-09-16T00:37:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e2659c640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 0x7f1e2810b450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e18031610 tx=0x7f1e180043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e1f7fe640 1 -- 172.21.15.102:0/4108232507 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1803f030 con 0x7f1e28100ad0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e281a2050 con 0x7f1e28100ad0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e1f7fe640 1 -- 172.21.15.102:0/4108232507 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e18003ce0 con 0x7f1e28100ad0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e1f7fe640 1 -- 172.21.15.102:0/4108232507 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18035070 con 0x7f1e28100ad0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.811+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e281a2550 con 0x7f1e28100ad0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.812+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e28107f10 con 0x7f1e28100ad0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.813+0000 7f1e1f7fe640 1 -- 172.21.15.102:0/4108232507 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1e18042030 con 0x7f1e28100ad0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.814+0000 7f1e1f7fe640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1e140a3240 0x7f1e140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.814+0000 7f1e1f7fe640 1 -- 172.21.15.102:0/4108232507 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6414+0+0 (secure 0 0 0) 0x7f1e180ef340 con 0x7f1e28100ad0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.814+0000 7f1e1ffff640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1e140a3240 0x7f1e140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.814+0000 7f1e1ffff640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1e140a3240 0x7f1e140a5700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1e2810b1d0 tx=0x7f1e1000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.818+0000 7f1e1f7fe640 1 -- 172.21.15.102:0/4108232507 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e180bc180 con 0x7f1e28100ad0 2024-09-16T00:37:33.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:33.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='osd.10 [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:33.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:33.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:33.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:33 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:33.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:33.993+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1e28062630 con 0x7f1e140a3240 2024-09-16T00:37:34.439 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:37:34.117+0000 7fc51987a640 -1 osd.10 0 waiting for initial osdmap 2024-09-16T00:37:34.439 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:37:34.130+0000 7fc51169a640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:37:34.731 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:37:34.731 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:37:34.731 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi081 2024-09-16T00:37:34.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi081 to 4272M 2024-09-16T00:37:34.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:37:34.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: osdmap e67: 11 total, 10 up, 11 in 2024-09-16T00:37:34.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:34.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:34.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:37:34.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:37:34.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:34.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:34 smithi102 ceph-mon[35364]: from='osd.10 ' entity='osd.10' 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi081 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi081 to 4272M 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: osdmap e67: 11 total, 10 up, 11 in 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:37:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:34.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:34 smithi081 ceph-mon[35539]: from='osd.10 ' entity='osd.10' 2024-09-16T00:37:34.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:37:34.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:37:34.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi081 2024-09-16T00:37:34.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi081 to 4272M 2024-09-16T00:37:34.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-16T00:37:34.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: osdmap e67: 11 total, 10 up, 11 in 2024-09-16T00:37:34.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:34.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:34.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:37:34.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:37:34.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:34.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:34 smithi031 ceph-mon[30956]: from='osd.10 ' entity='osd.10' 2024-09-16T00:37:35.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:35 smithi102 ceph-mon[35364]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:35.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:35 smithi102 ceph-mon[35364]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:35.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:35.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:35 smithi102 ceph-mon[35364]: osd.10 [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] boot 2024-09-16T00:37:35.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:35 smithi102 ceph-mon[35364]: osdmap e68: 11 total, 11 up, 11 in 2024-09-16T00:37:35.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:35 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:35.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:35 smithi102 ceph-mon[35364]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:37:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:35 smithi081 ceph-mon[35539]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:35 smithi081 ceph-mon[35539]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:35 smithi081 ceph-mon[35539]: osd.10 [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] boot 2024-09-16T00:37:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:35 smithi081 ceph-mon[35539]: osdmap e68: 11 total, 11 up, 11 in 2024-09-16T00:37:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:35 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:35 smithi081 ceph-mon[35539]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:37:35.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:35 smithi031 ceph-mon[30956]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:37:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:35 smithi031 ceph-mon[30956]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:35 smithi031 ceph-mon[30956]: osd.10 [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] boot 2024-09-16T00:37:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:35 smithi031 ceph-mon[30956]: osdmap e68: 11 total, 11 up, 11 in 2024-09-16T00:37:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:35 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:37:35.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:35 smithi031 ceph-mon[30956]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:37:36.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:36 smithi031 ceph-mon[30956]: osdmap e70: 11 total, 11 up, 11 in 2024-09-16T00:37:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:36 smithi031 ceph-mon[30956]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:36 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/3791498407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fddf945-ad39-4a5c-a40b-8ac262a7df75"}]: dispatch 2024-09-16T00:37:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:36 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fddf945-ad39-4a5c-a40b-8ac262a7df75"}]: dispatch 2024-09-16T00:37:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:36 smithi031 ceph-mon[30956]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fddf945-ad39-4a5c-a40b-8ac262a7df75"}]': finished 2024-09-16T00:37:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:36 smithi031 ceph-mon[30956]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T00:37:36.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:36 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:37:36.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:36 smithi081 ceph-mon[35539]: osdmap e70: 11 total, 11 up, 11 in 2024-09-16T00:37:36.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:36 smithi081 ceph-mon[35539]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:36.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:36 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/3791498407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fddf945-ad39-4a5c-a40b-8ac262a7df75"}]: dispatch 2024-09-16T00:37:36.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:36 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fddf945-ad39-4a5c-a40b-8ac262a7df75"}]: dispatch 2024-09-16T00:37:36.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:36 smithi081 ceph-mon[35539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fddf945-ad39-4a5c-a40b-8ac262a7df75"}]': finished 2024-09-16T00:37:36.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:36 smithi081 ceph-mon[35539]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T00:37:36.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:36 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:37:36.982 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:36 smithi102 ceph-mon[35364]: osdmap e70: 11 total, 11 up, 11 in 2024-09-16T00:37:36.982 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:36 smithi102 ceph-mon[35364]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:36.982 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:36 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/3791498407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fddf945-ad39-4a5c-a40b-8ac262a7df75"}]: dispatch 2024-09-16T00:37:36.982 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:36 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fddf945-ad39-4a5c-a40b-8ac262a7df75"}]: dispatch 2024-09-16T00:37:36.982 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:36 smithi102 ceph-mon[35364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fddf945-ad39-4a5c-a40b-8ac262a7df75"}]': finished 2024-09-16T00:37:36.983 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:36 smithi102 ceph-mon[35364]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T00:37:36.983 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:36 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:37:37.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:37 smithi031 ceph-mon[30956]: from='client.? 172.21.15.102:0/770933819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:37:37.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:37 smithi031 ceph-mon[30956]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:37:37.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:37 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:37:37.879 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:37 smithi081 ceph-mon[35539]: from='client.? 172.21.15.102:0/770933819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:37:37.879 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:37 smithi081 ceph-mon[35539]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:37:37.879 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:37 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:37:38.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:37 smithi102 ceph-mon[35364]: from='client.? 172.21.15.102:0/770933819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:37:38.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:37 smithi102 ceph-mon[35364]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:37:38.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:37 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:37:38.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:38 smithi031 ceph-mon[30956]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:38.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:38 smithi081 ceph-mon[35539]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:39.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:38 smithi102 ceph-mon[35364]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:40.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:40 smithi081 ceph-mon[35539]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:37:41.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:40 smithi102 ceph-mon[35364]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:37:41.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:40 smithi031 ceph-mon[30956]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:37:42.953 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:42 smithi102 ceph-mon[35364]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T00:37:42.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:42 smithi081 ceph-mon[35539]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T00:37:43.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:42 smithi031 ceph-mon[30956]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T00:37:44.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:44 smithi081 ceph-mon[35539]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:37:45.025 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:44 smithi102 ceph-mon[35364]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:37:45.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:44 smithi031 ceph-mon[30956]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:37:45.872 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:37:45.872 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:45.872 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:45 smithi102 ceph-mon[35364]: Deploying daemon osd.11 on smithi102 2024-09-16T00:37:45.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:37:45.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:45.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:45 smithi081 ceph-mon[35539]: Deploying daemon osd.11 on smithi102 2024-09-16T00:37:46.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:37:46.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:46.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:45 smithi031 ceph-mon[30956]: Deploying daemon osd.11 on smithi102 2024-09-16T00:37:46.973 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:46 smithi102 ceph-mon[35364]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-16T00:37:46.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:46 smithi081 ceph-mon[35539]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-16T00:37:47.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:46 smithi031 ceph-mon[30956]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-16T00:37:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:48 smithi081 ceph-mon[35539]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-16T00:37:49.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:48 smithi102 ceph-mon[35364]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-16T00:37:49.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:48 smithi031 ceph-mon[30956]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-16T00:37:50.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:50 smithi081 ceph-mon[35539]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:37:51.023 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:50 smithi102 ceph-mon[35364]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:37:51.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:50 smithi031 ceph-mon[30956]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:37:51.715 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:51.716 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:51.716 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:51 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:51.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:51.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:51.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:51 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:52.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:52.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:52.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:51 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:53.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:53 smithi102 ceph-mon[35364]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:53.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:53 smithi031 ceph-mon[30956]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:53.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:53 smithi081 ceph-mon[35539]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:55.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.160+0000 7f1e1f7fe640 1 -- 172.21.15.102:0/4108232507 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1e28062630 con 0x7f1e140a3240 2024-09-16T00:37:55.160 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 11 on host 'smithi102' 2024-09-16T00:37:55.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1e140a3240 msgr2=0x7f1e140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1e140a3240 0x7f1e140a5700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1e2810b1d0 tx=0x7f1e1000b390 comp rx=0 tx=0).stop 2024-09-16T00:37:55.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 msgr2=0x7f1e2810b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:55.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 0x7f1e2810b450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e18031610 tx=0x7f1e180043b0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 shutdown_connections 2024-09-16T00:37:55.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e28102690 0x7f1e28109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e28101cd0 0x7f1e28109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1e140a3240 0x7f1e140a5700 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 --2- 172.21.15.102:0/4108232507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e28100ad0 0x7f1e2810b450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:55.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.162+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 >> 172.21.15.102:0/4108232507 conn(0x7f1e280fc280 msgr2=0x7f1e281050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:55.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.163+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 shutdown_connections 2024-09-16T00:37:55.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:37:55.163+0000 7f1e2cc62640 1 -- 172.21.15.102:0/4108232507 wait complete. 2024-09-16T00:37:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:55 smithi102 ceph-mon[35364]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:55 smithi031 ceph-mon[30956]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.907 DEBUG:teuthology.orchestra.run.smithi102:osd.11> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.11.service 2024-09-16T00:37:55.909 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-16T00:37:55.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:37:55.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:55 smithi081 ceph-mon[35539]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:55.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:55.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:55.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:37:55.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:55.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:56.178 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:56.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 -- 172.21.15.31:0/941721741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5948104ff0 msgr2=0x7f59481073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 --2- 172.21.15.31:0/941721741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5948104ff0 0x7f59481073e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f593c009900 tx=0x7f593c02a410 comp rx=0 tx=0).stop 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 -- 172.21.15.31:0/941721741 shutdown_connections 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 --2- 172.21.15.31:0/941721741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5948107920 0x7f5948109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 --2- 172.21.15.31:0/941721741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5948104ff0 0x7f59481073e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 --2- 172.21.15.31:0/941721741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59481026c0 0x7f5948104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 -- 172.21.15.31:0/941721741 >> 172.21.15.31:0/941721741 conn(0x7f59480fc280 msgr2=0x7f59480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 -- 172.21.15.31:0/941721741 shutdown_connections 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 -- 172.21.15.31:0/941721741 wait complete. 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 Processor -- start 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.654+0000 7f594d6f7640 1 -- start start 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f594d6f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59481026c0 0x7f5948197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f594d6f7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5948104ff0 0x7f5948198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f594d6f7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5948107920 0x7f594819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f594d6f7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f594810d880 con 0x7f59481026c0 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f594d6f7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f594810d700 con 0x7f5948104ff0 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f594d6f7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f594810da00 con 0x7f5948107920 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f5946ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59481026c0 0x7f5948197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f5946ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59481026c0 0x7f5948197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43052/0 (socket says 172.21.15.31:43052) 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f5946ffd640 1 -- 172.21.15.31:0/2704877356 learned_addr learned my addr 172.21.15.31:0/2704877356 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f59477fe640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5948107920 0x7f594819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f59467fc640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5948104ff0 0x7f5948198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f5946ffd640 1 -- 172.21.15.31:0/2704877356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5948107920 msgr2=0x7f594819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f5946ffd640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5948107920 0x7f594819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f5946ffd640 1 -- 172.21.15.31:0/2704877356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5948104ff0 msgr2=0x7f5948198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f5946ffd640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5948104ff0 0x7f5948198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f5946ffd640 1 -- 172.21.15.31:0/2704877356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594819faf0 con 0x7f59481026c0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.655+0000 7f5946ffd640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59481026c0 0x7f5948197ec0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f593800e9f0 tx=0x7f593800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.656+0000 7f592ffff640 1 -- 172.21.15.31:0/2704877356 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593800cd80 con 0x7f59481026c0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.656+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594819fde0 con 0x7f59481026c0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.656+0000 7f592ffff640 1 -- 172.21.15.31:0/2704877356 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5938010910 con 0x7f59481026c0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.656+0000 7f592ffff640 1 -- 172.21.15.31:0/2704877356 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59380188f0 con 0x7f59481026c0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.656+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59481a0240 con 0x7f59481026c0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.657+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59480fdd70 con 0x7f59481026c0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.659+0000 7f592ffff640 1 -- 172.21.15.31:0/2704877356 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5938010ac0 con 0x7f59481026c0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.659+0000 7f592ffff640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f59300a3270 0x7f59300a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.660+0000 7f59467fc640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f59300a3270 0x7f59300a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:56.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.660+0000 7f592ffff640 1 -- 172.21.15.31:0/2704877356 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7f59380cfb30 con 0x7f59481026c0 2024-09-16T00:37:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.660+0000 7f59467fc640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f59300a3270 0x7f59300a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f593c02aa00 tx=0x7f593c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.662+0000 7f592ffff640 1 -- 172.21.15.31:0/2704877356 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f593809c740 con 0x7f59481026c0 2024-09-16T00:37:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.837+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f594802da10 con 0x7f59481026c0 2024-09-16T00:37:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.838+0000 7f592ffff640 1 -- 172.21.15.31:0/2704877356 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f59380a0a40 con 0x7f59481026c0 2024-09-16T00:37:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:37:56.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f59300a3270 msgr2=0x7f59300a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f59300a3270 0x7f59300a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f593c02aa00 tx=0x7f593c0315b0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59481026c0 msgr2=0x7f5948197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59481026c0 0x7f5948197ec0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f593800e9f0 tx=0x7f593800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 shutdown_connections 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f59300a3270 0x7f59300a5730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5948107920 0x7f594819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5948104ff0 0x7f5948198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 --2- 172.21.15.31:0/2704877356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59481026c0 0x7f5948197ec0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 >> 172.21.15.31:0/2704877356 conn(0x7f59480fc280 msgr2=0x7f5948103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 shutdown_connections 2024-09-16T00:37:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:56.842+0000 7f594d6f7640 1 -- 172.21.15.31:0/2704877356 wait complete. 2024-09-16T00:37:57.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:57 smithi031 ceph-mon[30956]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:57.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:57 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2704877356' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:37:57.743 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726447054,"num_in_osds":12,"osd_in_since":1726447056,"num_remapped_pgs":0} 2024-09-16T00:37:57.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:57 smithi102 ceph-mon[35364]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:57.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:57 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2704877356' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:37:57.800 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:37:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:37:57.535+0000 7f2e65d51640 -1 osd.11 0 log_to_monitors true 2024-09-16T00:37:57.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:57 smithi081 ceph-mon[35539]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:57.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:57 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2704877356' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:37:58.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:37:58.775 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:58 smithi031 ceph-mon[30956]: from='osd.11 [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:37:58.776 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:58 smithi031 ceph-mon[30956]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:37:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:58 smithi102 ceph-mon[35364]: from='osd.11 [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:37:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:58 smithi102 ceph-mon[35364]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:37:58.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:58 smithi081 ceph-mon[35539]: from='osd.11 [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:37:58.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:58 smithi081 ceph-mon[35539]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:37:59.020 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:37:59.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:59.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:37:59.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:37:59.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:37:59.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='osd.11 [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:37:59.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:37:59.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.624 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:37:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.619+0000 7f4afd42b640 1 -- 172.21.15.31:0/2462777291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8100490 msgr2=0x7f4af81028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.619+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2462777291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8100490 0x7f4af81028a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009900 tx=0x7f4ae802a410 comp rx=0 tx=0).stop 2024-09-16T00:37:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 -- 172.21.15.31:0/2462777291 shutdown_connections 2024-09-16T00:37:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2462777291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af8105790 0x7f4af8105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2462777291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4af8102de0 0x7f4af8105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2462777291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8100490 0x7f4af81028a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 -- 172.21.15.31:0/2462777291 >> 172.21.15.31:0/2462777291 conn(0x7f4af80fc2a0 msgr2=0x7f4af80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 -- 172.21.15.31:0/2462777291 shutdown_connections 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 -- 172.21.15.31:0/2462777291 wait complete. 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 Processor -- start 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 -- start start 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4af8100490 0x7f4af819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8102de0 0x7f4af819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af8105790 0x7f4af81a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8113ea0 con 0x7f4af8102de0 2024-09-16T00:37:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8113d20 con 0x7f4af8100490 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4afd42b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8114020 con 0x7f4af8105790 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.620+0000 7f4aeffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8102de0 0x7f4af819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4aeffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8102de0 0x7f4af819ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47700/0 (socket says 172.21.15.31:47700) 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4aeffff640 1 -- 172.21.15.31:0/2038940090 learned_addr learned my addr 172.21.15.31:0/2038940090 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4af77fe640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af8105790 0x7f4af81a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4af6ffd640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4af8100490 0x7f4af819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4aeffff640 1 -- 172.21.15.31:0/2038940090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af8105790 msgr2=0x7f4af81a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4aeffff640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af8105790 0x7f4af81a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4aeffff640 1 -- 172.21.15.31:0/2038940090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4af8100490 msgr2=0x7f4af819e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4aeffff640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4af8100490 0x7f4af819e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4aeffff640 1 -- 172.21.15.31:0/2038940090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af81a60f0 con 0x7f4af8102de0 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4aeffff640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8102de0 0x7f4af819ea00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000d950 tx=0x7f4ae000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4af4ff9640 1 -- 172.21.15.31:0/2038940090 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0017070 con 0x7f4af8102de0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af81a63e0 con 0x7f4af8102de0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4af4ff9640 1 -- 172.21.15.31:0/2038940090 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae0009c30 con 0x7f4af8102de0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4af4ff9640 1 -- 172.21.15.31:0/2038940090 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000f6b0 con 0x7f4af8102de0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.621+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af81a6970 con 0x7f4af8102de0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.622+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af80fdd90 con 0x7f4af8102de0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.624+0000 7f4af4ff9640 1 -- 172.21.15.31:0/2038940090 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4ae00109d0 con 0x7f4af8102de0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.625+0000 7f4af4ff9640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ae40a3270 0x7f4ae40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.625+0000 7f4af6ffd640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ae40a3270 0x7f4ae40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.625+0000 7f4af4ff9640 1 -- 172.21.15.31:0/2038940090 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6847+0+0 (secure 0 0 0) 0x7f4ae00cef30 con 0x7f4af8102de0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.625+0000 7f4af6ffd640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ae40a3270 0x7f4ae40a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8031610 tx=0x7f4ae8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.628+0000 7f4af4ff9640 1 -- 172.21.15.31:0/2038940090 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ae009c210 con 0x7f4af8102de0 2024-09-16T00:37:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='osd.11 [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:59.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:59.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:37:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.804+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4af802d9c0 con 0x7f4af8102de0 2024-09-16T00:37:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.804+0000 7f4af4ff9640 1 -- 172.21.15.31:0/2038940090 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f4ae009bb10 con 0x7f4af8102de0 2024-09-16T00:37:59.806 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:37:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.806+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ae40a3270 msgr2=0x7f4ae40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.806+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ae40a3270 0x7f4ae40a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8031610 tx=0x7f4ae8000bc0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.806+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8102de0 msgr2=0x7f4af819ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.806+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8102de0 0x7f4af819ea00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000d950 tx=0x7f4ae000de10 comp rx=0 tx=0).stop 2024-09-16T00:37:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.807+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 shutdown_connections 2024-09-16T00:37:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.807+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ae40a3270 0x7f4ae40a5730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.807+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af8105790 0x7f4af81a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.807+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8102de0 0x7f4af819ea00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.807+0000 7f4afd42b640 1 --2- 172.21.15.31:0/2038940090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4af8100490 0x7f4af819e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.807+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 >> 172.21.15.31:0/2038940090 conn(0x7f4af80fc2a0 msgr2=0x7f4af8103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.807+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 shutdown_connections 2024-09-16T00:37:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:37:59.807+0000 7f4afd42b640 1 -- 172.21.15.31:0/2038940090 wait complete. 2024-09-16T00:37:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:37:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:37:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:37:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='osd.11 [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:37:59.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:37:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:00.482 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726447054,"num_in_osds":12,"osd_in_since":1726447056,"num_remapped_pgs":0} 2024-09-16T00:38:00.526 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:38:00.212+0000 7f2e624e8640 -1 osd.11 0 waiting for initial osdmap 2024-09-16T00:38:00.526 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:38:00.227+0000 7f2e59af4640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:38:00.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:00 smithi031 ceph-mon[30956]: Detected new or changed devices on smithi102 2024-09-16T00:38:00.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:00 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi102 to 4272M 2024-09-16T00:38:00.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:00 smithi031 ceph-mon[30956]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:38:00.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:00 smithi031 ceph-mon[30956]: osdmap e74: 12 total, 11 up, 12 in 2024-09-16T00:38:00.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:00 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:00.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:00 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:00.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:00 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2038940090' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:38:00.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:00 smithi031 ceph-mon[30956]: from='osd.11 ' entity='osd.11' 2024-09-16T00:38:00.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-mon[35364]: Detected new or changed devices on smithi102 2024-09-16T00:38:00.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi102 to 4272M 2024-09-16T00:38:00.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-mon[35364]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:38:00.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-mon[35364]: osdmap e74: 12 total, 11 up, 12 in 2024-09-16T00:38:00.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:00.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:00.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2038940090' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:38:00.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:00 smithi102 ceph-mon[35364]: from='osd.11 ' entity='osd.11' 2024-09-16T00:38:00.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:00 smithi081 ceph-mon[35539]: Detected new or changed devices on smithi102 2024-09-16T00:38:00.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:00 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi102 to 4272M 2024-09-16T00:38:00.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:00 smithi081 ceph-mon[35539]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-16T00:38:00.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:00 smithi081 ceph-mon[35539]: osdmap e74: 12 total, 11 up, 12 in 2024-09-16T00:38:00.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:00 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:00.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:00 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:00.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:00 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2038940090' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:38:00.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:00 smithi081 ceph-mon[35539]: from='osd.11 ' entity='osd.11' 2024-09-16T00:38:01.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:38:01.753 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:01 smithi102 ceph-mon[35364]: purged_snaps scrub starts 2024-09-16T00:38:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:01 smithi102 ceph-mon[35364]: purged_snaps scrub ok 2024-09-16T00:38:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:01 smithi102 ceph-mon[35364]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:38:01.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:01.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:01 smithi102 ceph-mon[35364]: osd.11 [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] boot 2024-09-16T00:38:01.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:01 smithi102 ceph-mon[35364]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T00:38:01.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:01.874 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:01 smithi031 ceph-mon[30956]: purged_snaps scrub starts 2024-09-16T00:38:01.874 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:01 smithi031 ceph-mon[30956]: purged_snaps scrub ok 2024-09-16T00:38:01.874 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:01 smithi031 ceph-mon[30956]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:38:01.874 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:01.874 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:01 smithi031 ceph-mon[30956]: osd.11 [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] boot 2024-09-16T00:38:01.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:01 smithi031 ceph-mon[30956]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T00:38:01.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:01.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:01 smithi081 ceph-mon[35539]: purged_snaps scrub starts 2024-09-16T00:38:01.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:01 smithi081 ceph-mon[35539]: purged_snaps scrub ok 2024-09-16T00:38:01.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:01 smithi081 ceph-mon[35539]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:38:01.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:01.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:01 smithi081 ceph-mon[35539]: osd.11 [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] boot 2024-09-16T00:38:01.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:01 smithi081 ceph-mon[35539]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T00:38:01.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:38:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 -- 172.21.15.31:0/2147422041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc102e00 msgr2=0x7fc4cc103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/2147422041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc102e00 0x7fc4cc103280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009900 tx=0x7fc4c002a410 comp rx=0 tx=0).stop 2024-09-16T00:38:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 -- 172.21.15.31:0/2147422041 shutdown_connections 2024-09-16T00:38:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/2147422041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc1037c0 0x7fc4cc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/2147422041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc102e00 0x7fc4cc103280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/2147422041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4cc101c00 0x7fc4cc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 -- 172.21.15.31:0/2147422041 >> 172.21.15.31:0/2147422041 conn(0x7fc4cc0fd390 msgr2=0x7fc4cc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 -- 172.21.15.31:0/2147422041 shutdown_connections 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 -- 172.21.15.31:0/2147422041 wait complete. 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.228+0000 7fc4d25e6640 1 Processor -- start 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d25e6640 1 -- start start 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d25e6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4cc101c00 0x7fc4cc075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d25e6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102e00 0x7fc4cc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d25e6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc1037c0 0x7fc4cc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d25e6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc10d880 con 0x7fc4cc1037c0 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d25e6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc10d700 con 0x7fc4cc101c00 2024-09-16T00:38:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d25e6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc10da00 con 0x7fc4cc102e00 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d0b83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc1037c0 0x7fc4cc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d0b83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc1037c0 0x7fc4cc074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47728/0 (socket says 172.21.15.31:47728) 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d0b83640 1 -- 172.21.15.31:0/1433037762 learned_addr learned my addr 172.21.15.31:0/1433037762 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4cb7fe640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102e00 0x7fc4cc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4cbfff640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4cc101c00 0x7fc4cc075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d0b83640 1 -- 172.21.15.31:0/1433037762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102e00 msgr2=0x7fc4cc0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d0b83640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102e00 0x7fc4cc0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d0b83640 1 -- 172.21.15.31:0/1433037762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4cc101c00 msgr2=0x7fc4cc075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d0b83640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4cc101c00 0x7fc4cc075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.229+0000 7fc4d0b83640 1 -- 172.21.15.31:0/1433037762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc0753e0 con 0x7fc4cc1037c0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.230+0000 7fc4d0b83640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc1037c0 0x7fc4cc074b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400eaa0 tx=0x7fc4c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.230+0000 7fc4c97fa640 1 -- 172.21.15.31:0/1433037762 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400cdb0 con 0x7fc4cc1037c0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.230+0000 7fc4c97fa640 1 -- 172.21.15.31:0/1433037762 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4005020 con 0x7fc4cc1037c0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.230+0000 7fc4c97fa640 1 -- 172.21.15.31:0/1433037762 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40116b0 con 0x7fc4cc1037c0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.230+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc1a2060 con 0x7fc4cc1037c0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.230+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc1a2520 con 0x7fc4cc1037c0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.231+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4cc108fd0 con 0x7fc4cc1037c0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.232+0000 7fc4c97fa640 1 -- 172.21.15.31:0/1433037762 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc4c4005560 con 0x7fc4cc1037c0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.233+0000 7fc4c97fa640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc4b40a3240 0x7fc4b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.233+0000 7fc4cbfff640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc4b40a3240 0x7fc4b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.233+0000 7fc4c97fa640 1 -- 172.21.15.31:0/1433037762 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fc4c40cf610 con 0x7fc4cc1037c0 2024-09-16T00:38:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.233+0000 7fc4cbfff640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc4b40a3240 0x7fc4b40a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc102c60 tx=0x7fc4bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:02.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.237+0000 7fc4c97fa640 1 -- 172.21.15.31:0/1433037762 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4c4016030 con 0x7fc4cc1037c0 2024-09-16T00:38:02.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.412+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc4cc062df0 con 0x7fc4cc1037c0 2024-09-16T00:38:02.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.413+0000 7fc4c97fa640 1 -- 172.21.15.31:0/1433037762 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7fc4c409c180 con 0x7fc4cc1037c0 2024-09-16T00:38:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc4b40a3240 msgr2=0x7fc4b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc4b40a3240 0x7fc4b40a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc102c60 tx=0x7fc4bc00b390 comp rx=0 tx=0).stop 2024-09-16T00:38:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc1037c0 msgr2=0x7fc4cc074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc1037c0 0x7fc4cc074b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400eaa0 tx=0x7fc4c400ef60 comp rx=0 tx=0).stop 2024-09-16T00:38:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 shutdown_connections 2024-09-16T00:38:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4cc1037c0 0x7fc4cc074b20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc4b40a3240 0x7fc4b40a5700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102e00 0x7fc4cc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 --2- 172.21.15.31:0/1433037762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4cc101c00 0x7fc4cc075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 >> 172.21.15.31:0/1433037762 conn(0x7fc4cc0fd390 msgr2=0x7fc4cc1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 shutdown_connections 2024-09-16T00:38:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:02.416+0000 7fc4d25e6640 1 -- 172.21.15.31:0/1433037762 wait complete. 2024-09-16T00:38:03.336 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726447081,"num_in_osds":12,"osd_in_since":1726447056,"num_remapped_pgs":0} 2024-09-16T00:38:03.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:38:03.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:03 smithi081 ceph-mon[35539]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:03.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:03 smithi081 ceph-mon[35539]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:38:03.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:03 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1433037762' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:38:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:03 smithi102 ceph-mon[35364]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:03 smithi102 ceph-mon[35364]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:38:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:03 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1433037762' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:38:03.612 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:03.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:03 smithi031 ceph-mon[30956]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:03.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:03 smithi031 ceph-mon[30956]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:38:03.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:03 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1433037762' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:38:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.276+0000 7f932a056640 1 -- 172.21.15.31:0/514759682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324100a80 msgr2=0x7f9324100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.276+0000 7f932a056640 1 --2- 172.21.15.31:0/514759682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324100a80 0x7f9324100ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9318009930 tx=0x7f931802a420 comp rx=0 tx=0).stop 2024-09-16T00:38:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.276+0000 7f932a056640 1 -- 172.21.15.31:0/514759682 shutdown_connections 2024-09-16T00:38:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.276+0000 7f932a056640 1 --2- 172.21.15.31:0/514759682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9324101420 0x7f932410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.276+0000 7f932a056640 1 --2- 172.21.15.31:0/514759682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324100a80 0x7f9324100ee0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.276+0000 7f932a056640 1 --2- 172.21.15.31:0/514759682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9324106a80 0x7f9324106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.276+0000 7f932a056640 1 -- 172.21.15.31:0/514759682 >> 172.21.15.31:0/514759682 conn(0x7f93240fc7a0 msgr2=0x7f93240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.276+0000 7f932a056640 1 -- 172.21.15.31:0/514759682 shutdown_connections 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.276+0000 7f932a056640 1 -- 172.21.15.31:0/514759682 wait complete. 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f932a056640 1 Processor -- start 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f932a056640 1 -- start start 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f932a056640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9324100a80 0x7f9324198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f932a056640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324101420 0x7f93241989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f932a056640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9324106a80 0x7f932419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f932a056640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9324110e40 con 0x7f9324101420 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f932a056640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9324110cc0 con 0x7f9324106a80 2024-09-16T00:38:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f932a056640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9324110fc0 con 0x7f9324100a80 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f9322ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324101420 0x7f93241989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f9322ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324101420 0x7f93241989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47734/0 (socket says 172.21.15.31:47734) 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f9322ffd640 1 -- 172.21.15.31:0/222923578 learned_addr learned my addr 172.21.15.31:0/222923578 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f93237fe640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9324100a80 0x7f9324198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f9323fff640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9324106a80 0x7f932419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f9322ffd640 1 -- 172.21.15.31:0/222923578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9324100a80 msgr2=0x7f9324198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f9322ffd640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9324100a80 0x7f9324198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f9322ffd640 1 -- 172.21.15.31:0/222923578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9324106a80 msgr2=0x7f932419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f9322ffd640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9324106a80 0x7f932419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.277+0000 7f9322ffd640 1 -- 172.21.15.31:0/222923578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932419d390 con 0x7f9324101420 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.278+0000 7f9322ffd640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324101420 0x7f93241989c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f9318004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.278+0000 7f9320ff9640 1 -- 172.21.15.31:0/222923578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931803f030 con 0x7f9324101420 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.278+0000 7f9320ff9640 1 -- 172.21.15.31:0/222923578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9318003ad0 con 0x7f9324101420 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.278+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932419d620 con 0x7f9324101420 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.278+0000 7f9320ff9640 1 -- 172.21.15.31:0/222923578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318035070 con 0x7f9324101420 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.278+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93241a4f60 con 0x7f9324101420 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.279+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9324100ee0 con 0x7f9324101420 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.280+0000 7f9320ff9640 1 -- 172.21.15.31:0/222923578 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9318042030 con 0x7f9324101420 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.281+0000 7f9320ff9640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f930c0a3240 0x7f930c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.281+0000 7f9320ff9640 1 -- 172.21.15.31:0/222923578 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f93180efab0 con 0x7f9324101420 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.281+0000 7f93237fe640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f930c0a3240 0x7f930c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.281+0000 7f93237fe640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f930c0a3240 0x7f930c0a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f931400cac0 tx=0x7f931400c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.285+0000 7f9320ff9640 1 -- 172.21.15.31:0/222923578 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93180bcbd0 con 0x7f9324101420 2024-09-16T00:38:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.462+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93241a5130 con 0x7f9324101420 2024-09-16T00:38:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.464+0000 7f9320ff9640 1 -- 172.21.15.31:0/222923578 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18406 (secure 0 0 0) 0x7f93180bc4d0 con 0x7f9324101420 2024-09-16T00:38:04.464 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:04.464 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:38:02.217296+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:38:04.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f930c0a3240 msgr2=0x7f930c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:04.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f930c0a3240 0x7f930c0a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f931400cac0 tx=0x7f931400c4a0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324101420 msgr2=0x7f93241989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324101420 0x7f93241989c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f9318004270 comp rx=0 tx=0).stop 2024-09-16T00:38:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 shutdown_connections 2024-09-16T00:38:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9324106a80 0x7f932419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9324101420 0x7f93241989c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f930c0a3240 0x7f930c0a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 --2- 172.21.15.31:0/222923578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9324100a80 0x7f9324198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 >> 172.21.15.31:0/222923578 conn(0x7f93240fc7a0 msgr2=0x7f93241047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 shutdown_connections 2024-09-16T00:38:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:04.466+0000 7f932a056640 1 -- 172.21.15.31:0/222923578 wait complete. 2024-09-16T00:38:05.222 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:34:12.067708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T00:38:05.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T00:38:05.480 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:05 smithi081 ceph-mon[35539]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:05 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/222923578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:38:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:05 smithi102 ceph-mon[35364]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:05 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/222923578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:38:05.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:05 smithi031 ceph-mon[30956]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:05.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:05 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/222923578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:38:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 -- 172.21.15.31:0/237505342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 msgr2=0x7fb884100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 --2- 172.21.15.31:0/237505342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 0x7fb884100cb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb880009900 tx=0x7fb88002a410 comp rx=0 tx=0).stop 2024-09-16T00:38:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 -- 172.21.15.31:0/237505342 shutdown_connections 2024-09-16T00:38:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 --2- 172.21.15.31:0/237505342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88410e480 0x7fb884110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 --2- 172.21.15.31:0/237505342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb884101280 0x7fb88410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 --2- 172.21.15.31:0/237505342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 0x7fb884100cb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 -- 172.21.15.31:0/237505342 >> 172.21.15.31:0/237505342 conn(0x7fb8840fc7a0 msgr2=0x7fb8840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 -- 172.21.15.31:0/237505342 shutdown_connections 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 -- 172.21.15.31:0/237505342 wait complete. 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 Processor -- start 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.981+0000 7fb88c051640 1 -- start start 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb88c051640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 0x7fb88419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb88c051640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb884101280 0x7fb88419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb88c051640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88410e480 0x7fb88419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb88c051640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb884114250 con 0x7fb8841008d0 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb88c051640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841140d0 con 0x7fb88410e480 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb88c051640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841143d0 con 0x7fb884101280 2024-09-16T00:38:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb889ded640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 0x7fb88419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb889ded640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 0x7fb88419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47746/0 (socket says 172.21.15.31:47746) 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb889ded640 1 -- 172.21.15.31:0/1903694964 learned_addr learned my addr 172.21.15.31:0/1903694964 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb88a5ee640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88410e480 0x7fb88419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb889ded640 1 -- 172.21.15.31:0/1903694964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb884101280 msgr2=0x7fb88419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb889ded640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb884101280 0x7fb88419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb889ded640 1 -- 172.21.15.31:0/1903694964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88410e480 msgr2=0x7fb88419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb889ded640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88410e480 0x7fb88419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb889ded640 1 -- 172.21.15.31:0/1903694964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88419f6e0 con 0x7fb8841008d0 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb889ded640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 0x7fb88419a730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb880006010 tx=0x7fb880003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb87affd640 1 -- 172.21.15.31:0/1903694964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88003f070 con 0x7fb8841008d0 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb87affd640 1 -- 172.21.15.31:0/1903694964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880031c90 con 0x7fb8841008d0 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.982+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88419f970 con 0x7fb8841008d0 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.983+0000 7fb87affd640 1 -- 172.21.15.31:0/1903694964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880035030 con 0x7fb8841008d0 2024-09-16T00:38:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.983+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8841a72b0 con 0x7fb8841008d0 2024-09-16T00:38:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.983+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88406b070 con 0x7fb8841008d0 2024-09-16T00:38:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.985+0000 7fb87affd640 1 -- 172.21.15.31:0/1903694964 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb880042070 con 0x7fb8841008d0 2024-09-16T00:38:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.985+0000 7fb87affd640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8700a3240 0x7fb8700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.986+0000 7fb8895ec640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8700a3240 0x7fb8700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.986+0000 7fb87affd640 1 -- 172.21.15.31:0/1903694964 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fb8800f0390 con 0x7fb8841008d0 2024-09-16T00:38:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.986+0000 7fb8895ec640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8700a3240 0x7fb8700a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb88410ed80 tx=0x7fb874006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:05.989+0000 7fb87affd640 1 -- 172.21.15.31:0/1903694964 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8800bd4b0 con 0x7fb8841008d0 2024-09-16T00:38:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.167+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb884109610 con 0x7fb8841008d0 2024-09-16T00:38:06.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.168+0000 7fb87affd640 1 -- 172.21.15.31:0/1903694964 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7fb8800bcdb0 con 0x7fb8841008d0 2024-09-16T00:38:06.168 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-16T00:38:06.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8700a3240 msgr2=0x7fb8700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8700a3240 0x7fb8700a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb88410ed80 tx=0x7fb874006c90 comp rx=0 tx=0).stop 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 msgr2=0x7fb88419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 0x7fb88419a730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb880006010 tx=0x7fb880003ba0 comp rx=0 tx=0).stop 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 shutdown_connections 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88410e480 0x7fb88419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8700a3240 0x7fb8700a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb884101280 0x7fb88419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 --2- 172.21.15.31:0/1903694964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8841008d0 0x7fb88419a730 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 >> 172.21.15.31:0/1903694964 conn(0x7fb8840fc7a0 msgr2=0x7fb8840feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 shutdown_connections 2024-09-16T00:38:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:06.170+0000 7fb88c051640 1 -- 172.21.15.31:0/1903694964 wait complete. 2024-09-16T00:38:06.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:06 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1903694964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:38:06.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:06 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1903694964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:38:06.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:06 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1903694964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:38:06.896 INFO:tasks.cephadm:Adding mds.a on smithi031 2024-09-16T00:38:06.896 INFO:tasks.cephadm:Adding mds.d on smithi031 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.g on smithi031 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.j on smithi031 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.b on smithi081 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.e on smithi081 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.h on smithi081 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.k on smithi081 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.c on smithi102 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.f on smithi102 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.i on smithi102 2024-09-16T00:38:06.897 INFO:tasks.cephadm:Adding mds.l on smithi102 2024-09-16T00:38:06.897 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l' 2024-09-16T00:38:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:38:07.482 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:07 smithi102 ceph-mon[35364]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:07.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:07 smithi081 ceph-mon[35539]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:07 smithi031 ceph-mon[30956]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.662+0000 7fe9c35e2640 1 -- 172.21.15.102:0/3120382872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 msgr2=0x7fe9bc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.662+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/3120382872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 0x7fe9bc100ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009900 tx=0x7fe9b802a410 comp rx=0 tx=0).stop 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.662+0000 7fe9c35e2640 1 -- 172.21.15.102:0/3120382872 shutdown_connections 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.662+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/3120382872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9bc102690 0x7fe9bc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.662+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/3120382872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9bc101cd0 0x7fe9bc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.662+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/3120382872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 0x7fe9bc100ed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.662+0000 7fe9c35e2640 1 -- 172.21.15.102:0/3120382872 >> 172.21.15.102:0/3120382872 conn(0x7fe9bc0fc280 msgr2=0x7fe9bc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.662+0000 7fe9c35e2640 1 -- 172.21.15.102:0/3120382872 shutdown_connections 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.662+0000 7fe9c35e2640 1 -- 172.21.15.102:0/3120382872 wait complete. 2024-09-16T00:38:07.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c35e2640 1 Processor -- start 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c35e2640 1 -- start start 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c35e2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 0x7fe9bc06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c35e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9bc101cd0 0x7fe9bc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c35e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9bc102690 0x7fe9bc19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c35e2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc10c7b0 con 0x7fe9bc101cd0 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c35e2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc10c630 con 0x7fe9bc102690 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c35e2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc10c930 con 0x7fe9bc100ad0 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c137e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 0x7fe9bc06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c137e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 0x7fe9bc06aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34998/0 (socket says 172.21.15.102:34998) 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c137e640 1 -- 172.21.15.102:0/75899149 learned_addr learned my addr 172.21.15.102:0/75899149 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c0b7d640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9bc101cd0 0x7fe9bc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c1b7f640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9bc102690 0x7fe9bc19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:07.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c137e640 1 -- 172.21.15.102:0/75899149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9bc102690 msgr2=0x7fe9bc19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c137e640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9bc102690 0x7fe9bc19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c137e640 1 -- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9bc101cd0 msgr2=0x7fe9bc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c137e640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9bc101cd0 0x7fe9bc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c137e640 1 -- 172.21.15.102:0/75899149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc19fbc0 con 0x7fe9bc100ad0 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.663+0000 7fe9c137e640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 0x7fe9bc06aad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8031610 tx=0x7fe9b80043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.664+0000 7fe9b27fc640 1 -- 172.21.15.102:0/75899149 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b803f030 con 0x7fe9bc100ad0 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.664+0000 7fe9b27fc640 1 -- 172.21.15.102:0/75899149 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b8003ce0 con 0x7fe9bc100ad0 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.664+0000 7fe9b27fc640 1 -- 172.21.15.102:0/75899149 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8035070 con 0x7fe9bc100ad0 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.664+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc19fdf0 con 0x7fe9bc100ad0 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.664+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9bc1a02d0 con 0x7fe9bc100ad0 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.665+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9bc107ef0 con 0x7fe9bc100ad0 2024-09-16T00:38:07.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.667+0000 7fe9b27fc640 1 -- 172.21.15.102:0/75899149 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe9b8042030 con 0x7fe9bc100ad0 2024-09-16T00:38:07.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.667+0000 7fe9b27fc640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9a80a32c0 0x7fe9a80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:07.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.668+0000 7fe9b27fc640 1 -- 172.21.15.102:0/75899149 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe9b80efd00 con 0x7fe9bc100ad0 2024-09-16T00:38:07.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.668+0000 7fe9c0b7d640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9a80a32c0 0x7fe9a80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:07.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.668+0000 7fe9c0b7d640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9a80a32c0 0x7fe9a80a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc06b050 tx=0x7fe9ac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:07.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.671+0000 7fe9b27fc640 1 -- 172.21.15.102:0/75899149 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9b80bcec0 con 0x7fe9bc100ad0 2024-09-16T00:38:07.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.853+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}) -- 0x7fe9bc0627f0 con 0x7fe9a80a32c0 2024-09-16T00:38:07.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.861+0000 7fe9b27fc640 1 -- 172.21.15.102:0/75899149 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe9bc0627f0 con 0x7fe9a80a32c0 2024-09-16T00:38:07.861 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mds.all update... 2024-09-16T00:38:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9a80a32c0 msgr2=0x7fe9a80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9a80a32c0 0x7fe9a80a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc06b050 tx=0x7fe9ac008040 comp rx=0 tx=0).stop 2024-09-16T00:38:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 msgr2=0x7fe9bc06aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 0x7fe9bc06aad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8031610 tx=0x7fe9b80043b0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 shutdown_connections 2024-09-16T00:38:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9bc102690 0x7fe9bc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9bc101cd0 0x7fe9bc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9a80a32c0 0x7fe9a80a5780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 --2- 172.21.15.102:0/75899149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9bc100ad0 0x7fe9bc06aad0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:07.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 >> 172.21.15.102:0/75899149 conn(0x7fe9bc0fc280 msgr2=0x7fe9bc1050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:07.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 shutdown_connections 2024-09-16T00:38:07.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T00:38:07.864+0000 7fe9c35e2640 1 -- 172.21.15.102:0/75899149 wait complete. 2024-09-16T00:38:08.596 DEBUG:teuthology.orchestra.run.smithi031:mds.a> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.a.service 2024-09-16T00:38:08.599 DEBUG:teuthology.orchestra.run.smithi031:mds.d> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.d.service 2024-09-16T00:38:08.642 DEBUG:teuthology.orchestra.run.smithi031:mds.g> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.g.service 2024-09-16T00:38:08.685 DEBUG:teuthology.orchestra.run.smithi031:mds.j> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.j.service 2024-09-16T00:38:08.728 DEBUG:teuthology.orchestra.run.smithi081:mds.b> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.b.service 2024-09-16T00:38:08.731 DEBUG:teuthology.orchestra.run.smithi081:mds.e> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.e.service 2024-09-16T00:38:08.777 DEBUG:teuthology.orchestra.run.smithi081:mds.h> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.h.service 2024-09-16T00:38:08.820 DEBUG:teuthology.orchestra.run.smithi081:mds.k> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.k.service 2024-09-16T00:38:08.863 DEBUG:teuthology.orchestra.run.smithi102:mds.c> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.c.service 2024-09-16T00:38:08.866 DEBUG:teuthology.orchestra.run.smithi102:mds.f> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.f.service 2024-09-16T00:38:08.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:08.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:08.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-09-16T00:38:08.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:08.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:08.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:08.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:08.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:08.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:08.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:08.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:08.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:08 smithi031 ceph-mon[30956]: Deploying daemon mds.l on smithi102 2024-09-16T00:38:08.909 DEBUG:teuthology.orchestra.run.smithi102:mds.i> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.i.service 2024-09-16T00:38:08.952 DEBUG:teuthology.orchestra.run.smithi102:mds.l> sudo journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.l.service 2024-09-16T00:38:08.995 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-16T00:38:08.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:09.170 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:09.170 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:09.170 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-09-16T00:38:09.170 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:09.170 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:09.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:09.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:09.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:09.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:09.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:09.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:09.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:08 smithi102 ceph-mon[35364]: Deploying daemon mds.l on smithi102 2024-09-16T00:38:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-09-16T00:38:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:09.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:09.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:09.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:09.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:09.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:08 smithi081 ceph-mon[35539]: Deploying daemon mds.l on smithi102 2024-09-16T00:38:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.858+0000 7f7575cc2640 1 -- 172.21.15.31:0/2664946322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 msgr2=0x7f75701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.858+0000 7f7575cc2640 1 --2- 172.21.15.31:0/2664946322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 0x7f75701021c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7560009900 tx=0x7f756002a410 comp rx=0 tx=0).stop 2024-09-16T00:38:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.858+0000 7f7575cc2640 1 -- 172.21.15.31:0/2664946322 shutdown_connections 2024-09-16T00:38:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.858+0000 7f7575cc2640 1 --2- 172.21.15.31:0/2664946322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7570106f10 0x7f757010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.858+0000 7f7575cc2640 1 --2- 172.21.15.31:0/2664946322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570102790 0x7f75701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.858+0000 7f7575cc2640 1 --2- 172.21.15.31:0/2664946322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 0x7f75701021c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.858+0000 7f7575cc2640 1 -- 172.21.15.31:0/2664946322 >> 172.21.15.31:0/2664946322 conn(0x7f75700fd8c0 msgr2=0x7f75700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.858+0000 7f7575cc2640 1 -- 172.21.15.31:0/2664946322 shutdown_connections 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.858+0000 7f7575cc2640 1 -- 172.21.15.31:0/2664946322 wait complete. 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.859+0000 7f7575cc2640 1 Processor -- start 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.859+0000 7f7575cc2640 1 -- start start 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.859+0000 7f7575cc2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 0x7f75700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.859+0000 7f7575cc2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570102790 0x7f7570074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.859+0000 7f7575cc2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7570106f10 0x7f7570074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.859+0000 7f7575cc2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7570111f90 con 0x7f7570101de0 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.859+0000 7f7575cc2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7570111e10 con 0x7f7570102790 2024-09-16T00:38:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.859+0000 7f7575cc2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7570112110 con 0x7f7570106f10 2024-09-16T00:38:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.859+0000 7f756f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 0x7f75700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 0x7f75700744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39126/0 (socket says 172.21.15.31:39126) 2024-09-16T00:38:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756f7fe640 1 -- 172.21.15.31:0/3722978234 learned_addr learned my addr 172.21.15.31:0/3722978234 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756effd640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570102790 0x7f7570074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756ffff640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7570106f10 0x7f7570074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756f7fe640 1 -- 172.21.15.31:0/3722978234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7570106f10 msgr2=0x7f7570074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756f7fe640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7570106f10 0x7f7570074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756f7fe640 1 -- 172.21.15.31:0/3722978234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570102790 msgr2=0x7f7570074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756f7fe640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570102790 0x7f7570074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756f7fe640 1 -- 172.21.15.31:0/3722978234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75701a6920 con 0x7f7570101de0 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.860+0000 7f756f7fe640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 0x7f75700744f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7560005ab0 tx=0x7f75600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.862+0000 7f756cff9640 1 -- 172.21.15.31:0/3722978234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756003f030 con 0x7f7570101de0 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.862+0000 7f756cff9640 1 -- 172.21.15.31:0/3722978234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7560003ce0 con 0x7f7570101de0 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.862+0000 7f756cff9640 1 -- 172.21.15.31:0/3722978234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560035070 con 0x7f7570101de0 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.862+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75701a6bb0 con 0x7f7570101de0 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.862+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a7090 con 0x7f7570101de0 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.864+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75701021c0 con 0x7f7570101de0 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.864+0000 7f756cff9640 1 -- 172.21.15.31:0/3722978234 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7560042030 con 0x7f7570101de0 2024-09-16T00:38:09.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.865+0000 7f756cff9640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75580a3240 0x7f75580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.865+0000 7f756effd640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75580a3240 0x7f75580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.865+0000 7f756cff9640 1 -- 172.21.15.31:0/3722978234 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f75600efd30 con 0x7f7570101de0 2024-09-16T00:38:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.866+0000 7f756effd640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75580a3240 0x7f75580a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75701087d0 tx=0x7f7564009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:09.869+0000 7f756cff9640 1 -- 172.21.15.31:0/3722978234 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75600bcef0 con 0x7f7570101de0 2024-09-16T00:38:10.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.079+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f757010a0f0 con 0x7f7570101de0 2024-09-16T00:38:10.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.080+0000 7f756cff9640 1 -- 172.21.15.31:0/3722978234 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f75600bc7f0 con 0x7f7570101de0 2024-09-16T00:38:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:10.080 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":1,"btime":"2024-09-16T00:31:04:619302+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-16T00:38:10.081 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 1 2024-09-16T00:38:10.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.082+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75580a3240 msgr2=0x7f75580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:10.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75580a3240 0x7f75580a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75701087d0 tx=0x7f7564009110 comp rx=0 tx=0).stop 2024-09-16T00:38:10.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 msgr2=0x7f75700744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:10.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 0x7f75700744f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7560005ab0 tx=0x7f75600042f0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 shutdown_connections 2024-09-16T00:38:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75580a3240 0x7f75580a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7570106f10 0x7f7570074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570102790 0x7f7570074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 --2- 172.21.15.31:0/3722978234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7570101de0 0x7f75700744f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 >> 172.21.15.31:0/3722978234 conn(0x7f75700fd8c0 msgr2=0x7f75700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 shutdown_connections 2024-09-16T00:38:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:10.083+0000 7f7575cc2640 1 -- 172.21.15.31:0/3722978234 wait complete. 2024-09-16T00:38:10.549 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:38:10 smithi102 systemd[1]: Starting Ceph mds.l for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:10.914 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:10 smithi102 ceph-mon[35364]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:10.914 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:10 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3722978234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:10.914 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:38:10 smithi102 podman[64386]: 2024-09-16 00:38:10.525963566 +0000 UTC m=+0.015667106 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:10.914 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:38:10 smithi102 podman[64386]: 2024-09-16 00:38:10.653050563 +0000 UTC m=+0.142754088 container create c7327996730350fabecfd8bf3d8d1e113a22bfe9eafcd115fcd117390e194f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:38:10.914 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:38:10 smithi102 podman[64386]: 2024-09-16 00:38:10.91457468 +0000 UTC m=+0.404278204 container init c7327996730350fabecfd8bf3d8d1e113a22bfe9eafcd115fcd117390e194f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:38:11.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:10 smithi031 ceph-mon[30956]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:11.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:10 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3722978234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:11.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:10 smithi081 ceph-mon[35539]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:11.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:10 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3722978234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:11.299 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:38:10 smithi102 podman[64386]: 2024-09-16 00:38:10.917901891 +0000 UTC m=+0.407605424 container start c7327996730350fabecfd8bf3d8d1e113a22bfe9eafcd115fcd117390e194f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T00:38:11.299 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:38:10 smithi102 ceph-mds[64405]: -- 172.21.15.102:0/1604892888 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557689d5d860 con 0x557689d3b000 2024-09-16T00:38:11.299 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:38:10 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l[64401]: starting mds.l at 2024-09-16T00:38:11.300 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:38:11 smithi102 bash[64386]: c7327996730350fabecfd8bf3d8d1e113a22bfe9eafcd115fcd117390e194f21 2024-09-16T00:38:11.300 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:38:11 smithi102 systemd[1]: Started Ceph mds.l for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:12.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.102:6832/316287545,v1:172.21.15.102:6833/316287545] up:boot 2024-09-16T00:38:12.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: fsmap 1 up:standby 2024-09-16T00:38:12.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:38:12.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:12.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:12.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:12.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:12.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:12.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:12.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:11 smithi081 ceph-mon[35539]: Deploying daemon mds.k on smithi081 2024-09-16T00:38:12.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.102:6832/316287545,v1:172.21.15.102:6833/316287545] up:boot 2024-09-16T00:38:12.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: fsmap 1 up:standby 2024-09-16T00:38:12.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:38:12.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:12.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:12.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:12.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:12.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:12.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:12.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:11 smithi102 ceph-mon[35364]: Deploying daemon mds.k on smithi081 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.102:6832/316287545,v1:172.21.15.102:6833/316287545] up:boot 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: fsmap 1 up:standby 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:11 smithi031 ceph-mon[30956]: Deploying daemon mds.k on smithi081 2024-09-16T00:38:12.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:13.100 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:38:13 smithi081 systemd[1]: Starting Ceph mds.k for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:13.282 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:13 smithi031 ceph-mon[30956]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:13.414 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:13 smithi081 ceph-mon[35539]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:13.414 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:38:13 smithi081 podman[64631]: 2024-09-16 00:38:13.127406497 +0000 UTC m=+0.015877800 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:13.414 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:38:13 smithi081 podman[64631]: 2024-09-16 00:38:13.284000507 +0000 UTC m=+0.172471812 container create 616001c7e847b63845c02613cc7d84c9f4b5bd4e46968c91698c2154ae5016d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T00:38:13.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:13 smithi102 ceph-mon[35364]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:13.747 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:38:13 smithi081 podman[64631]: 2024-09-16 00:38:13.527685945 +0000 UTC m=+0.416157251 container init 616001c7e847b63845c02613cc7d84c9f4b5bd4e46968c91698c2154ae5016d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-k, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T00:38:13.747 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:38:13 smithi081 podman[64631]: 2024-09-16 00:38:13.530436156 +0000 UTC m=+0.418907460 container start 616001c7e847b63845c02613cc7d84c9f4b5bd4e46968c91698c2154ae5016d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T00:38:13.748 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:38:13 smithi081 ceph-mds[64650]: -- 172.21.15.81:0/1925525388 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ff1cad860 con 0x561ff1c8b400 2024-09-16T00:38:13.748 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:38:13 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-k[64646]: starting mds.k at 2024-09-16T00:38:13.748 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:38:13 smithi081 bash[64631]: 616001c7e847b63845c02613cc7d84c9f4b5bd4e46968c91698c2154ae5016d4 2024-09-16T00:38:13.748 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:38:13 smithi081 systemd[1]: Started Ceph mds.k for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:13.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.801+0000 7fc856b3d640 1 -- 172.21.15.31:0/990687747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8501037a0 msgr2=0x7fc85010bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:13.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.801+0000 7fc856b3d640 1 --2- 172.21.15.31:0/990687747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8501037a0 0x7fc85010bc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc840009ca0 tx=0x7fc84002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:13.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.801+0000 7fc856b3d640 1 -- 172.21.15.31:0/990687747 shutdown_connections 2024-09-16T00:38:13.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.801+0000 7fc856b3d640 1 --2- 172.21.15.31:0/990687747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85010c2f0 0x7fc85010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:13.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.801+0000 7fc856b3d640 1 --2- 172.21.15.31:0/990687747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8501037a0 0x7fc85010bc00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:13.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.801+0000 7fc856b3d640 1 --2- 172.21.15.31:0/990687747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc850102df0 0x7fc8501031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:13.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.801+0000 7fc856b3d640 1 -- 172.21.15.31:0/990687747 >> 172.21.15.31:0/990687747 conn(0x7fc8500fc7d0 msgr2=0x7fc8500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.801+0000 7fc856b3d640 1 -- 172.21.15.31:0/990687747 shutdown_connections 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.801+0000 7fc856b3d640 1 -- 172.21.15.31:0/990687747 wait complete. 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc856b3d640 1 Processor -- start 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc856b3d640 1 -- start start 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc856b3d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc850102df0 0x7fc8501a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc856b3d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8501037a0 0x7fc8501a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc856b3d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85010c2f0 0x7fc85019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc856b3d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc8501120d0 con 0x7fc850102df0 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc856b3d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc850111f50 con 0x7fc85010c2f0 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc856b3d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc850112250 con 0x7fc8501037a0 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc8548d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc850102df0 0x7fc8501a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:13.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc8548d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc850102df0 0x7fc8501a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39150/0 (socket says 172.21.15.31:39150) 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc8548d9640 1 -- 172.21.15.31:0/3625235986 learned_addr learned my addr 172.21.15.31:0/3625235986 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc8550da640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85010c2f0 0x7fc85019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc84ffff640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8501037a0 0x7fc8501a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc8548d9640 1 -- 172.21.15.31:0/3625235986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8501037a0 msgr2=0x7fc8501a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc8548d9640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8501037a0 0x7fc8501a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc8548d9640 1 -- 172.21.15.31:0/3625235986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85010c2f0 msgr2=0x7fc85019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc8548d9640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85010c2f0 0x7fc85019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc8548d9640 1 -- 172.21.15.31:0/3625235986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85019aee0 con 0x7fc850102df0 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.802+0000 7fc84ffff640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8501037a0 0x7fc8501a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.803+0000 7fc8548d9640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc850102df0 0x7fc8501a05a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc83800e9f0 tx=0x7fc83800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.803+0000 7fc84dffb640 1 -- 172.21.15.31:0/3625235986 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83800cd80 con 0x7fc850102df0 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.803+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85019b1d0 con 0x7fc850102df0 2024-09-16T00:38:13.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.803+0000 7fc84dffb640 1 -- 172.21.15.31:0/3625235986 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc838005500 con 0x7fc850102df0 2024-09-16T00:38:13.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.803+0000 7fc84dffb640 1 -- 172.21.15.31:0/3625235986 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838010690 con 0x7fc850102df0 2024-09-16T00:38:13.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.803+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8501a7520 con 0x7fc850102df0 2024-09-16T00:38:13.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.804+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85006a790 con 0x7fc850102df0 2024-09-16T00:38:13.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.806+0000 7fc84dffb640 1 -- 172.21.15.31:0/3625235986 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc8380046c0 con 0x7fc850102df0 2024-09-16T00:38:13.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.806+0000 7fc84dffb640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc83c0a3240 0x7fc83c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:13.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.807+0000 7fc84ffff640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc83c0a3240 0x7fc83c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:13.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.807+0000 7fc84dffb640 1 -- 172.21.15.31:0/3625235986 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fc8380cfbd0 con 0x7fc850102df0 2024-09-16T00:38:13.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.807+0000 7fc84ffff640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc83c0a3240 0x7fc83c0a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc840006010 tx=0x7fc8400093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:13.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:13.810+0000 7fc84dffb640 1 -- 172.21.15.31:0/3625235986 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8380541b0 con 0x7fc850102df0 2024-09-16T00:38:14.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.026+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc85019bba0 con 0x7fc850102df0 2024-09-16T00:38:14.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.027+0000 7fc84dffb640 1 -- 172.21.15.31:0/3625235986 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7fc83809ccf0 con 0x7fc850102df0 2024-09-16T00:38:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"btime":"2024-09-16T00:38:10:945374+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T00:38:14.028 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.029+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc83c0a3240 msgr2=0x7fc83c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.029+0000 7fc856b3d640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc83c0a3240 0x7fc83c0a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc840006010 tx=0x7fc8400093f0 comp rx=0 tx=0).stop 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.029+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc850102df0 msgr2=0x7fc8501a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.029+0000 7fc856b3d640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc850102df0 0x7fc8501a05a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc83800e9f0 tx=0x7fc83800edf0 comp rx=0 tx=0).stop 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.029+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 shutdown_connections 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.029+0000 7fc856b3d640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc85010c2f0 0x7fc85019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.029+0000 7fc856b3d640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc83c0a3240 0x7fc83c0a5700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.029+0000 7fc856b3d640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8501037a0 0x7fc8501a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.029+0000 7fc856b3d640 1 --2- 172.21.15.31:0/3625235986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc850102df0 0x7fc8501a05a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.030+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 >> 172.21.15.31:0/3625235986 conn(0x7fc8500fc7d0 msgr2=0x7fc85010d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:14.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.030+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 shutdown_connections 2024-09-16T00:38:14.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:14.030+0000 7fc856b3d640 1 -- 172.21.15.31:0/3625235986 wait complete. 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: Deploying daemon mds.j on smithi031 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3625235986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:14.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.81:6832/1467259017,v1:172.21.15.81:6833/1467259017] up:boot 2024-09-16T00:38:14.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: fsmap 2 up:standby 2024-09-16T00:38:14.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:14 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:38:14.942 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:14.942 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:14.942 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:14.942 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:14.942 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:14.942 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:14.943 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:14.943 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: Deploying daemon mds.j on smithi031 2024-09-16T00:38:14.943 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3625235986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:14.943 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.81:6832/1467259017,v1:172.21.15.81:6833/1467259017] up:boot 2024-09-16T00:38:14.943 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: fsmap 2 up:standby 2024-09-16T00:38:14.943 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:14 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:38:15.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: Deploying daemon mds.j on smithi031 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3625235986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.81:6832/1467259017,v1:172.21.15.81:6833/1467259017] up:boot 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: fsmap 2 up:standby 2024-09-16T00:38:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:14 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:38:16.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:16 smithi031 ceph-mon[30956]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:16.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:16 smithi081 ceph-mon[35539]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:17.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:16 smithi102 ceph-mon[35364]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:17.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:17.334 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:17.457 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:38:17 smithi031 systemd[1]: Started Ceph mds.j for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.819+0000 7f157e96a640 1 -- 172.21.15.31:0/2888387600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578100e20 msgr2=0x7f1578101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.819+0000 7f157e96a640 1 --2- 172.21.15.31:0/2888387600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578100e20 0x7f1578101200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1568009900 tx=0x7f156802a480 comp rx=0 tx=0).stop 2024-09-16T00:38:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.819+0000 7f157e96a640 1 -- 172.21.15.31:0/2888387600 shutdown_connections 2024-09-16T00:38:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.819+0000 7f157e96a640 1 --2- 172.21.15.31:0/2888387600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578102ef0 0x7f157810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.819+0000 7f157e96a640 1 --2- 172.21.15.31:0/2888387600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1578102570 0x7f15781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.819+0000 7f157e96a640 1 --2- 172.21.15.31:0/2888387600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578100e20 0x7f1578101200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.819+0000 7f157e96a640 1 -- 172.21.15.31:0/2888387600 >> 172.21.15.31:0/2888387600 conn(0x7f15780fc7d0 msgr2=0x7f15780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.819+0000 7f157e96a640 1 -- 172.21.15.31:0/2888387600 shutdown_connections 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.819+0000 7f157e96a640 1 -- 172.21.15.31:0/2888387600 wait complete. 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157e96a640 1 Processor -- start 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157e96a640 1 -- start start 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157e96a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578100e20 0x7f15780fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157e96a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1578102570 0x7f15780fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157e96a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578102ef0 0x7f15780fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157e96a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1578111020 con 0x7f1578102ef0 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157e96a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1578110ea0 con 0x7f1578100e20 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157e96a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f15781111a0 con 0x7f1578102570 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157cf07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578102ef0 0x7f15780fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157cf07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578102ef0 0x7f15780fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33748/0 (socket says 172.21.15.31:33748) 2024-09-16T00:38:17.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157cf07640 1 -- 172.21.15.31:0/1692343658 learned_addr learned my addr 172.21.15.31:0/1692343658 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f15777fe640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1578102570 0x7f15780fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f1577fff640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578100e20 0x7f15780fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157cf07640 1 -- 172.21.15.31:0/1692343658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1578102570 msgr2=0x7f15780fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157cf07640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1578102570 0x7f15780fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157cf07640 1 -- 172.21.15.31:0/1692343658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578100e20 msgr2=0x7f15780fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157cf07640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578100e20 0x7f15780fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f157cf07640 1 -- 172.21.15.31:0/1692343658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15781a6b30 con 0x7f1578102ef0 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.820+0000 7f1577fff640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578100e20 0x7f15780fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.821+0000 7f157cf07640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578102ef0 0x7f15780fd700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f157000b970 tx=0x7f157000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.821+0000 7f15757fa640 1 -- 172.21.15.31:0/1692343658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570019070 con 0x7f1578102ef0 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.821+0000 7f15757fa640 1 -- 172.21.15.31:0/1692343658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1570014440 con 0x7f1578102ef0 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.821+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15781a6e20 con 0x7f1578102ef0 2024-09-16T00:38:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.821+0000 7f15757fa640 1 -- 172.21.15.31:0/1692343658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15700075d0 con 0x7f1578102ef0 2024-09-16T00:38:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.821+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15781a72e0 con 0x7f1578102ef0 2024-09-16T00:38:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.822+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1578101280 con 0x7f1578102ef0 2024-09-16T00:38:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.823+0000 7f15757fa640 1 -- 172.21.15.31:0/1692343658 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f157001c070 con 0x7f1578102ef0 2024-09-16T00:38:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.824+0000 7f15757fa640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f15600a3240 0x7f15600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.824+0000 7f1577fff640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f15600a3240 0x7f15600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.824+0000 7f15757fa640 1 -- 172.21.15.31:0/1692343658 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f15700ceb40 con 0x7f1578102ef0 2024-09-16T00:38:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.824+0000 7f1577fff640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f15600a3240 0x7f15600a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1568000c00 tx=0x7f15680041e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:17.828+0000 7f15757fa640 1 -- 172.21.15.31:0/1692343658 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f157009bc60 con 0x7f1578102ef0 2024-09-16T00:38:18.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.048+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15780fe1b0 con 0x7f1578102ef0 2024-09-16T00:38:18.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.049+0000 7f15757fa640 1 -- 172.21.15.31:0/1692343658 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f15780fe1b0 con 0x7f1578102ef0 2024-09-16T00:38:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:18.050 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"btime":"2024-09-16T00:38:17:680375+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T00:38:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2024-09-16T00:38:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f15600a3240 msgr2=0x7f15600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f15600a3240 0x7f15600a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1568000c00 tx=0x7f15680041e0 comp rx=0 tx=0).stop 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578102ef0 msgr2=0x7f15780fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578102ef0 0x7f15780fd700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f157000b970 tx=0x7f157000bd70 comp rx=0 tx=0).stop 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 shutdown_connections 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1578102ef0 0x7f15780fd700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f15600a3240 0x7f15600a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1578102570 0x7f15780fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 --2- 172.21.15.31:0/1692343658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578100e20 0x7f15780fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 >> 172.21.15.31:0/1692343658 conn(0x7f15780fc7d0 msgr2=0x7f157806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 shutdown_connections 2024-09-16T00:38:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:18.051+0000 7f157e96a640 1 -- 172.21.15.31:0/1692343658 wait complete. 2024-09-16T00:38:18.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:18.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:18.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:18.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:18.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:18.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:18.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: Deploying daemon mds.d on smithi031 2024-09-16T00:38:18.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.31:6834/3753760611,v1:172.21.15.31:6835/3753760611] up:boot 2024-09-16T00:38:18.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: fsmap 3 up:standby 2024-09-16T00:38:18.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:38:18.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:18 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1692343658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:18.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:18.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:18.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:18.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:18.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:18.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:18.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: Deploying daemon mds.d on smithi031 2024-09-16T00:38:18.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.31:6834/3753760611,v1:172.21.15.31:6835/3753760611] up:boot 2024-09-16T00:38:18.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: fsmap 3 up:standby 2024-09-16T00:38:18.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:38:18.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:18 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1692343658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: Deploying daemon mds.d on smithi031 2024-09-16T00:38:18.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.31:6834/3753760611,v1:172.21.15.31:6835/3753760611] up:boot 2024-09-16T00:38:18.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: fsmap 3 up:standby 2024-09-16T00:38:18.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:38:18.801 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:18 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1692343658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:19.617 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:19 smithi031 ceph-mon[30956]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:19.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:19 smithi081 ceph-mon[35539]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:19 smithi102 ceph-mon[35364]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:19.875 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:38:19 smithi031 systemd[1]: Starting Ceph mds.d for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:20.317 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:38:20 smithi031 podman[68262]: 2024-09-16 00:38:20.202879808 +0000 UTC m=+0.105289874 container create 71e2abc1939901d9ab872feda504ad59c65a480e79dd98700a9484f54bf406e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T00:38:20.317 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:38:20 smithi031 podman[68262]: 2024-09-16 00:38:20.113827255 +0000 UTC m=+0.016237322 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:20.625 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:38:20 smithi031 podman[68262]: 2024-09-16 00:38:20.430867579 +0000 UTC m=+0.333277646 container init 71e2abc1939901d9ab872feda504ad59c65a480e79dd98700a9484f54bf406e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:38:20.626 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:38:20 smithi031 podman[68262]: 2024-09-16 00:38:20.433907323 +0000 UTC m=+0.336317390 container start 71e2abc1939901d9ab872feda504ad59c65a480e79dd98700a9484f54bf406e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:38:20.626 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:38:20 smithi031 ceph-mds[68281]: -- 172.21.15.31:0/22686030 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593f1a15860 con 0x5593f19f3000 2024-09-16T00:38:20.626 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:38:20 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-d[68277]: starting mds.d at 2024-09-16T00:38:20.626 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:38:20 smithi031 bash[68262]: 71e2abc1939901d9ab872feda504ad59c65a480e79dd98700a9484f54bf406e0 2024-09-16T00:38:20.626 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:38:20 smithi031 systemd[1]: Started Ceph mds.d for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:21.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:21 smithi031 ceph-mon[30956]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:21.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:21 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:21.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:21 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:21.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:21 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:21.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:21 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:21.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:21 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:21.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:21 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:21.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:21 smithi081 ceph-mon[35539]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:21.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:21 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:21.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:21 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:21.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:21 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:21.720 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:21 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:21.720 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:21 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:21.720 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:21 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:21.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:21 smithi102 ceph-mon[35364]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:21.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:21 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:21.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:21 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:21.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:21 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:21.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:21 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:21.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:21 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:21.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:21 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:22.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:22.325 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:22.412 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:22 smithi081 ceph-mon[35539]: Deploying daemon mds.e on smithi081 2024-09-16T00:38:22.412 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:22 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.31:6836/3116907098,v1:172.21.15.31:6837/3116907098] up:boot 2024-09-16T00:38:22.412 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:22 smithi081 ceph-mon[35539]: fsmap 4 up:standby 2024-09-16T00:38:22.412 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:22 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:38:22.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:22 smithi102 ceph-mon[35364]: Deploying daemon mds.e on smithi081 2024-09-16T00:38:22.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:22 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.31:6836/3116907098,v1:172.21.15.31:6837/3116907098] up:boot 2024-09-16T00:38:22.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:22 smithi102 ceph-mon[35364]: fsmap 4 up:standby 2024-09-16T00:38:22.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:22 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:38:22.593 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:22 smithi031 ceph-mon[30956]: Deploying daemon mds.e on smithi081 2024-09-16T00:38:22.593 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:22 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.31:6836/3116907098,v1:172.21.15.31:6837/3116907098] up:boot 2024-09-16T00:38:22.594 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:22 smithi031 ceph-mon[30956]: fsmap 4 up:standby 2024-09-16T00:38:22.594 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:22 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:38:22.747 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:38:22 smithi081 systemd[1]: Starting Ceph mds.e for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:22.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.808+0000 7f2d1f954640 1 -- 172.21.15.31:0/1997426238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 msgr2=0x7f2d18110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:22.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.808+0000 7f2d1f954640 1 --2- 172.21.15.31:0/1997426238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 0x7f2d18110870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009ca0 tx=0x7f2d1002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:22.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.808+0000 7f2d1f954640 1 -- 172.21.15.31:0/1997426238 shutdown_connections 2024-09-16T00:38:22.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.808+0000 7f2d1f954640 1 --2- 172.21.15.31:0/1997426238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 0x7f2d18110870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:22.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.808+0000 7f2d1f954640 1 --2- 172.21.15.31:0/1997426238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d18101280 0x7f2d1810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:22.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.808+0000 7f2d1f954640 1 --2- 172.21.15.31:0/1997426238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d181008d0 0x7f2d18100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:22.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.808+0000 7f2d1f954640 1 -- 172.21.15.31:0/1997426238 >> 172.21.15.31:0/1997426238 conn(0x7f2d180fc7a0 msgr2=0x7f2d180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.808+0000 7f2d1f954640 1 -- 172.21.15.31:0/1997426238 shutdown_connections 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1f954640 1 -- 172.21.15.31:0/1997426238 wait complete. 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1f954640 1 Processor -- start 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1f954640 1 -- start start 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1f954640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d181008d0 0x7f2d1819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1f954640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d18101280 0x7f2d1819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1f954640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 0x7f2d1819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1f954640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18114160 con 0x7f2d1810e480 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1f954640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18113fe0 con 0x7f2d181008d0 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1f954640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181142e0 con 0x7f2d18101280 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1def1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 0x7f2d1819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1def1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 0x7f2d1819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33770/0 (socket says 172.21.15.31:33770) 2024-09-16T00:38:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.809+0000 7f2d1def1640 1 -- 172.21.15.31:0/3417014110 learned_addr learned my addr 172.21.15.31:0/3417014110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1d6f0640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d181008d0 0x7f2d1819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1ceef640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d18101280 0x7f2d1819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1def1640 1 -- 172.21.15.31:0/3417014110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d18101280 msgr2=0x7f2d1819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1def1640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d18101280 0x7f2d1819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1def1640 1 -- 172.21.15.31:0/3417014110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d181008d0 msgr2=0x7f2d1819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1def1640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d181008d0 0x7f2d1819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1def1640 1 -- 172.21.15.31:0/3417014110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1819f5b0 con 0x7f2d1810e480 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1ceef640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d18101280 0x7f2d1819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1def1640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 0x7f2d1819eed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2d1002ade0 tx=0x7f2d100319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d0a7fc640 1 -- 172.21.15.31:0/3417014110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10003b70 con 0x7f2d1810e480 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d0a7fc640 1 -- 172.21.15.31:0/3417014110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d10003d10 con 0x7f2d1810e480 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d0a7fc640 1 -- 172.21.15.31:0/3417014110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1002fcf0 con 0x7f2d1810e480 2024-09-16T00:38:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1819f840 con 0x7f2d1810e480 2024-09-16T00:38:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.810+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181a7180 con 0x7f2d1810e480 2024-09-16T00:38:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.811+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d18100cb0 con 0x7f2d1810e480 2024-09-16T00:38:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.812+0000 7f2d0a7fc640 1 -- 172.21.15.31:0/3417014110 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2d10004050 con 0x7f2d1810e480 2024-09-16T00:38:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.813+0000 7f2d0a7fc640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d040a32c0 0x7f2d040a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.813+0000 7f2d1d6f0640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d040a32c0 0x7f2d040a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.813+0000 7f2d0a7fc640 1 -- 172.21.15.31:0/3417014110 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f2d100f0a60 con 0x7f2d1810e480 2024-09-16T00:38:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.813+0000 7f2d1d6f0640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d040a32c0 0x7f2d040a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400abe0 tx=0x7f2d1400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:22.816+0000 7f2d0a7fc640 1 -- 172.21.15.31:0/3417014110 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d1007f1b0 con 0x7f2d1810e480 2024-09-16T00:38:23.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.026+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d1819b960 con 0x7f2d1810e480 2024-09-16T00:38:23.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.026+0000 7f2d0a7fc640 1 -- 172.21.15.31:0/3417014110 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f2d1819b960 con 0x7f2d1810e480 2024-09-16T00:38:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:23.027 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"btime":"2024-09-16T00:38:21:330809+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T00:38:23.027 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 5 2024-09-16T00:38:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d040a32c0 msgr2=0x7f2d040a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d040a32c0 0x7f2d040a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400abe0 tx=0x7f2d1400a2d0 comp rx=0 tx=0).stop 2024-09-16T00:38:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 msgr2=0x7f2d1819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 0x7f2d1819eed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2d1002ade0 tx=0x7f2d100319f0 comp rx=0 tx=0).stop 2024-09-16T00:38:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 shutdown_connections 2024-09-16T00:38:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d1810e480 0x7f2d1819eed0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d040a32c0 0x7f2d040a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d18101280 0x7f2d1819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 --2- 172.21.15.31:0/3417014110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d181008d0 0x7f2d1819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:23.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 >> 172.21.15.31:0/3417014110 conn(0x7f2d180fc7a0 msgr2=0x7f2d180feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:23.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 shutdown_connections 2024-09-16T00:38:23.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:23.028+0000 7f2d1f954640 1 -- 172.21.15.31:0/3417014110 wait complete. 2024-09-16T00:38:23.061 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:38:22 smithi081 podman[64942]: 2024-09-16 00:38:22.713060988 +0000 UTC m=+0.015809460 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:23.061 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:38:22 smithi081 podman[64942]: 2024-09-16 00:38:22.814352802 +0000 UTC m=+0.117101266 container create df6645692f3317be96e7f1e9d277a3340ec199207fc122ee84f9936c186bfe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T00:38:23.061 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:38:23 smithi081 podman[64942]: 2024-09-16 00:38:23.058791297 +0000 UTC m=+0.361539761 container init df6645692f3317be96e7f1e9d277a3340ec199207fc122ee84f9936c186bfe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:38:23.062 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:38:23 smithi081 podman[64942]: 2024-09-16 00:38:23.061857378 +0000 UTC m=+0.364605842 container start df6645692f3317be96e7f1e9d277a3340ec199207fc122ee84f9936c186bfe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:38:23.350 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-mds[64961]: -- 172.21.15.81:0/4054668392 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3dd46d860 con 0x55a3dd44a800 2024-09-16T00:38:23.350 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e[64957]: starting mds.e at 2024-09-16T00:38:23.350 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:38:23 smithi081 bash[64942]: df6645692f3317be96e7f1e9d277a3340ec199207fc122ee84f9936c186bfe77 2024-09-16T00:38:23.350 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:38:23 smithi081 systemd[1]: Started Ceph mds.e for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:23.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:23 smithi031 ceph-mon[30956]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:23.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:23 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3417014110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:23.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:23 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:23.673 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:23 smithi102 ceph-mon[35364]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:23.673 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:23 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3417014110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:23.673 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:23.673 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:23.673 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:23.673 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:23.673 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:23.674 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:23 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-mon[35539]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3417014110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:23 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:24.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:24 smithi031 ceph-mon[30956]: Deploying daemon mds.f on smithi102 2024-09-16T00:38:24.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:24 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.81:6834/1522041920,v1:172.21.15.81:6835/1522041920] up:boot 2024-09-16T00:38:24.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:24 smithi031 ceph-mon[30956]: fsmap 5 up:standby 2024-09-16T00:38:24.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:24 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:38:24.693 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:24 smithi102 ceph-mon[35364]: Deploying daemon mds.f on smithi102 2024-09-16T00:38:24.693 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:24 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.81:6834/1522041920,v1:172.21.15.81:6835/1522041920] up:boot 2024-09-16T00:38:24.693 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:24 smithi102 ceph-mon[35364]: fsmap 5 up:standby 2024-09-16T00:38:24.694 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:24 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:38:24.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:24 smithi081 ceph-mon[35539]: Deploying daemon mds.f on smithi102 2024-09-16T00:38:24.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:24 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.81:6834/1522041920,v1:172.21.15.81:6835/1522041920] up:boot 2024-09-16T00:38:24.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:24 smithi081 ceph-mon[35539]: fsmap 5 up:standby 2024-09-16T00:38:24.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:24 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:38:25.299 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:38:25 smithi102 systemd[1]: Starting Ceph mds.f for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:25.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:25 smithi031 ceph-mon[30956]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:25.662 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:25 smithi102 ceph-mon[35364]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:25.662 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:38:25 smithi102 podman[64690]: 2024-09-16 00:38:25.289305609 +0000 UTC m=+0.015664944 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:25.662 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:38:25 smithi102 podman[64690]: 2024-09-16 00:38:25.409534219 +0000 UTC m=+0.135893544 container create 99fe3278e5e6ed360037b61bc3847aafec1d3f685c7a2480df7e24cc459f6df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-f, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T00:38:25.662 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:38:25 smithi102 podman[64690]: 2024-09-16 00:38:25.662416499 +0000 UTC m=+0.388775822 container init 99fe3278e5e6ed360037b61bc3847aafec1d3f685c7a2480df7e24cc459f6df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T00:38:25.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:25.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:25 smithi081 ceph-mon[35539]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:25.982 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:26.049 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:38:25 smithi102 podman[64690]: 2024-09-16 00:38:25.665749842 +0000 UTC m=+0.392109167 container start 99fe3278e5e6ed360037b61bc3847aafec1d3f685c7a2480df7e24cc459f6df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:38:26.049 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:38:25 smithi102 ceph-mds[64709]: -- 172.21.15.102:0/4098052817 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f363bd9860 con 0x55f363bb7000 2024-09-16T00:38:26.050 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:38:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-f[64705]: starting mds.f at 2024-09-16T00:38:26.050 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:38:25 smithi102 bash[64690]: 99fe3278e5e6ed360037b61bc3847aafec1d3f685c7a2480df7e24cc459f6df4 2024-09-16T00:38:26.050 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:38:25 smithi102 systemd[1]: Started Ceph mds.f for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:26.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.499+0000 7fc687e2d640 1 -- 172.21.15.31:0/2957840515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68010e500 msgr2=0x7fc680110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:26.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.499+0000 7fc687e2d640 1 --2- 172.21.15.31:0/2957840515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68010e500 0x7fc680110910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc678009ca0 tx=0x7fc67802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.500+0000 7fc687e2d640 1 -- 172.21.15.31:0/2957840515 shutdown_connections 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.500+0000 7fc687e2d640 1 --2- 172.21.15.31:0/2957840515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68010e500 0x7fc680110910 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.500+0000 7fc687e2d640 1 --2- 172.21.15.31:0/2957840515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc680101440 0x7fc68010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.500+0000 7fc687e2d640 1 --2- 172.21.15.31:0/2957840515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc680100b20 0x7fc680100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.500+0000 7fc687e2d640 1 -- 172.21.15.31:0/2957840515 >> 172.21.15.31:0/2957840515 conn(0x7fc6800fc7d0 msgr2=0x7fc6800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.500+0000 7fc687e2d640 1 -- 172.21.15.31:0/2957840515 shutdown_connections 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.500+0000 7fc687e2d640 1 -- 172.21.15.31:0/2957840515 wait complete. 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.500+0000 7fc687e2d640 1 Processor -- start 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.500+0000 7fc687e2d640 1 -- start start 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc687e2d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc680100b20 0x7fc6801a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc687e2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc680101440 0x7fc6801a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc687e2d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc68010e500 0x7fc68019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc687e2d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801142a0 con 0x7fc680101440 2024-09-16T00:38:26.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc687e2d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc680114120 con 0x7fc680100b20 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc687e2d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc680114420 con 0x7fc68010e500 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc6853c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc680101440 0x7fc6801a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc685bc9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc680100b20 0x7fc6801a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc685bc9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc680100b20 0x7fc6801a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:50350/0 (socket says 172.21.15.31:50350) 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc685bc9640 1 -- 172.21.15.31:0/677910609 learned_addr learned my addr 172.21.15.31:0/677910609 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc6863ca640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc68010e500 0x7fc68019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc685bc9640 1 -- 172.21.15.31:0/677910609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc68010e500 msgr2=0x7fc68019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc685bc9640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc68010e500 0x7fc68019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc685bc9640 1 -- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc680101440 msgr2=0x7fc6801a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc685bc9640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc680101440 0x7fc6801a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc685bc9640 1 -- 172.21.15.31:0/677910609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68019ace0 con 0x7fc680100b20 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.501+0000 7fc6853c8640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc680101440 0x7fc6801a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:38:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.502+0000 7fc685bc9640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc680100b20 0x7fc6801a03f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00d770 tx=0x7fc67c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.502+0000 7fc676ffd640 1 -- 172.21.15.31:0/677910609 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c017070 con 0x7fc680100b20 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.502+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68019afd0 con 0x7fc680100b20 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.502+0000 7fc676ffd640 1 -- 172.21.15.31:0/677910609 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c005500 con 0x7fc680100b20 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.502+0000 7fc676ffd640 1 -- 172.21.15.31:0/677910609 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c004e00 con 0x7fc680100b20 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.502+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6801a72c0 con 0x7fc680100b20 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.503+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc680100f00 con 0x7fc680100b20 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.504+0000 7fc676ffd640 1 -- 172.21.15.31:0/677910609 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc67c004fa0 con 0x7fc680100b20 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.505+0000 7fc676ffd640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc66c0a3240 0x7fc66c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.505+0000 7fc6853c8640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc66c0a3240 0x7fc66c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.505+0000 7fc676ffd640 1 -- 172.21.15.31:0/677910609 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fc67c0ceec0 con 0x7fc680100b20 2024-09-16T00:38:26.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.505+0000 7fc6853c8640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc66c0a3240 0x7fc66c0a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc6801027b0 tx=0x7fc67000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.508+0000 7fc676ffd640 1 -- 172.21.15.31:0/677910609 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc67c09ba30 con 0x7fc680100b20 2024-09-16T00:38:26.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.720+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6801a7490 con 0x7fc680100b20 2024-09-16T00:38:26.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.721+0000 7fc676ffd640 1 -- 172.21.15.31:0/677910609 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7fc67c09fd30 con 0x7fc680100b20 2024-09-16T00:38:26.722 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:26.722 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"btime":"2024-09-16T00:38:26:355037+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-16T00:38:26.722 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-09-16T00:38:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc66c0a3240 msgr2=0x7fc66c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc66c0a3240 0x7fc66c0a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc6801027b0 tx=0x7fc67000b410 comp rx=0 tx=0).stop 2024-09-16T00:38:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc680100b20 msgr2=0x7fc6801a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc680100b20 0x7fc6801a03f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00d770 tx=0x7fc67c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:38:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 shutdown_connections 2024-09-16T00:38:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc66c0a3240 0x7fc66c0a5700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc68010e500 0x7fc68019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc680101440 0x7fc6801a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 --2- 172.21.15.31:0/677910609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc680100b20 0x7fc6801a03f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:26.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.723+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 >> 172.21.15.31:0/677910609 conn(0x7fc6800fc7d0 msgr2=0x7fc6800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:26.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.724+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 shutdown_connections 2024-09-16T00:38:26.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:26.724+0000 7fc687e2d640 1 -- 172.21.15.31:0/677910609 wait complete. 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: Deploying daemon mds.c on smithi102 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.102:6834/1825124874,v1:172.21.15.102:6835/1825124874] up:boot 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: fsmap 6 up:standby 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:38:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:26 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/677910609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:27.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:27.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: Deploying daemon mds.c on smithi102 2024-09-16T00:38:27.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.102:6834/1825124874,v1:172.21.15.102:6835/1825124874] up:boot 2024-09-16T00:38:27.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: fsmap 6 up:standby 2024-09-16T00:38:27.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:38:27.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:26 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/677910609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:27.266 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:27.266 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:27.266 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:27.266 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:27.267 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:27.267 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:27.267 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:27.267 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: Deploying daemon mds.c on smithi102 2024-09-16T00:38:27.267 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.102:6834/1825124874,v1:172.21.15.102:6835/1825124874] up:boot 2024-09-16T00:38:27.267 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: fsmap 6 up:standby 2024-09-16T00:38:27.267 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:38:27.267 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:26 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/677910609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:28.002 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:38:27 smithi102 systemd[1]: Starting Ceph mds.c for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:28.277 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:38:28 smithi102 podman[64990]: 2024-09-16 00:38:27.904779647 +0000 UTC m=+0.015483855 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:28.277 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:38:28 smithi102 podman[64990]: 2024-09-16 00:38:28.032835688 +0000 UTC m=+0.143539892 container create 95a1cc22f1c2628f4485f174f3a84fe7b07d5d84acdc0c3cbe05d6acbe2edf0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-c, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:38:28.277 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:38:28 smithi102 podman[64990]: 2024-09-16 00:38:28.277346657 +0000 UTC m=+0.388050860 container init 95a1cc22f1c2628f4485f174f3a84fe7b07d5d84acdc0c3cbe05d6acbe2edf0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-c, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:38:28.549 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:38:28 smithi102 podman[64990]: 2024-09-16 00:38:28.280222302 +0000 UTC m=+0.390926507 container start 95a1cc22f1c2628f4485f174f3a84fe7b07d5d84acdc0c3cbe05d6acbe2edf0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T00:38:28.549 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:38:28 smithi102 ceph-mds[65009]: -- 172.21.15.102:0/970129462 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56089473d860 con 0x56089471b400 2024-09-16T00:38:28.549 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:38:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-c[65005]: starting mds.c at 2024-09-16T00:38:28.549 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:38:28 smithi102 bash[64990]: 95a1cc22f1c2628f4485f174f3a84fe7b07d5d84acdc0c3cbe05d6acbe2edf0c 2024-09-16T00:38:28.549 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:38:28 smithi102 systemd[1]: Started Ceph mds.c for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:29.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:29.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: Deploying daemon mds.h on smithi081 2024-09-16T00:38:29.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.102:6836/272579536,v1:172.21.15.102:6837/272579536] up:boot 2024-09-16T00:38:29.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: fsmap 7 up:standby 2024-09-16T00:38:29.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:29 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:38:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: Deploying daemon mds.h on smithi081 2024-09-16T00:38:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.102:6836/272579536,v1:172.21.15.102:6837/272579536] up:boot 2024-09-16T00:38:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: fsmap 7 up:standby 2024-09-16T00:38:29.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:29 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:38:29.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: Deploying daemon mds.h on smithi081 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.102:6836/272579536,v1:172.21.15.102:6837/272579536] up:boot 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: fsmap 7 up:standby 2024-09-16T00:38:29.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:29 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:38:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.159+0000 7f93a45b4640 1 -- 172.21.15.31:0/3810232129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 msgr2=0x7f939c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.159+0000 7f93a45b4640 1 --2- 172.21.15.31:0/3810232129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 0x7f939c1067e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f938c009900 tx=0x7f938c02a410 comp rx=0 tx=0).stop 2024-09-16T00:38:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.159+0000 7f93a45b4640 1 -- 172.21.15.31:0/3810232129 shutdown_connections 2024-09-16T00:38:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.159+0000 7f93a45b4640 1 --2- 172.21.15.31:0/3810232129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939c106f10 0x7f939c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.159+0000 7f93a45b4640 1 --2- 172.21.15.31:0/3810232129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 0x7f939c1067e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.160+0000 7f93a45b4640 1 --2- 172.21.15.31:0/3810232129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f939c101de0 0x7f939c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.160+0000 7f93a45b4640 1 -- 172.21.15.31:0/3810232129 >> 172.21.15.31:0/3810232129 conn(0x7f939c0fd8c0 msgr2=0x7f939c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.160+0000 7f93a45b4640 1 -- 172.21.15.31:0/3810232129 shutdown_connections 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.160+0000 7f93a45b4640 1 -- 172.21.15.31:0/3810232129 wait complete. 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.160+0000 7f93a45b4640 1 Processor -- start 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.160+0000 7f93a45b4640 1 -- start start 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.160+0000 7f93a45b4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939c101de0 0x7f939c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.160+0000 7f93a45b4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 0x7f939c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.160+0000 7f93a45b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f939c106f10 0x7f939c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a45b4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f939c111f70 con 0x7f939c102790 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a45b4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f939c111df0 con 0x7f939c106f10 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a45b4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1120f0 con 0x7f939c101de0 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a1b4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 0x7f939c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a1b4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 0x7f939c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52360/0 (socket says 172.21.15.31:52360) 2024-09-16T00:38:30.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a1b4f640 1 -- 172.21.15.31:0/1875206638 learned_addr learned my addr 172.21.15.31:0/1875206638 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a2350640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939c101de0 0x7f939c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a2b51640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f939c106f10 0x7f939c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a1b4f640 1 -- 172.21.15.31:0/1875206638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939c101de0 msgr2=0x7f939c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a1b4f640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939c101de0 0x7f939c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a1b4f640 1 -- 172.21.15.31:0/1875206638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f939c106f10 msgr2=0x7f939c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a1b4f640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f939c106f10 0x7f939c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a1b4f640 1 -- 172.21.15.31:0/1875206638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c1a6930 con 0x7f939c102790 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a1b4f640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 0x7f939c074a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f938c006010 tx=0x7f938c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93937fe640 1 -- 172.21.15.31:0/1875206638 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c03f030 con 0x7f939c102790 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93937fe640 1 -- 172.21.15.31:0/1875206638 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938c003ce0 con 0x7f939c102790 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c1a6bc0 con 0x7f939c102790 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93937fe640 1 -- 172.21.15.31:0/1875206638 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c035070 con 0x7f939c102790 2024-09-16T00:38:30.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.161+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c1a70f0 con 0x7f939c102790 2024-09-16T00:38:30.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.162+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f939c1021c0 con 0x7f939c102790 2024-09-16T00:38:30.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.164+0000 7f93937fe640 1 -- 172.21.15.31:0/1875206638 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f938c042030 con 0x7f939c102790 2024-09-16T00:38:30.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.164+0000 7f93937fe640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93880a32c0 0x7f93880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:30.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.164+0000 7f93a2350640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93880a32c0 0x7f93880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:30.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.164+0000 7f93937fe640 1 -- 172.21.15.31:0/1875206638 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f938c0efae0 con 0x7f939c102790 2024-09-16T00:38:30.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.165+0000 7f93a2350640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93880a32c0 0x7f93880a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9398009bd0 tx=0x7f9398005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:30.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.168+0000 7f93937fe640 1 -- 172.21.15.31:0/1875206638 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f938c0bc710 con 0x7f939c102790 2024-09-16T00:38:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.377+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f939c10a150 con 0x7f939c102790 2024-09-16T00:38:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.377+0000 7f93937fe640 1 -- 172.21.15.31:0/1875206638 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7f938c0c0a10 con 0x7f939c102790 2024-09-16T00:38:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:30.378 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"btime":"2024-09-16T00:38:28:813553+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-16T00:38:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2024-09-16T00:38:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93880a32c0 msgr2=0x7f93880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93880a32c0 0x7f93880a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9398009bd0 tx=0x7f9398005f90 comp rx=0 tx=0).stop 2024-09-16T00:38:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 msgr2=0x7f939c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 0x7f939c074a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f938c006010 tx=0x7f938c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 shutdown_connections 2024-09-16T00:38:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f939c106f10 0x7f939c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f939c102790 0x7f939c074a30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93880a32c0 0x7f93880a5780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 --2- 172.21.15.31:0/1875206638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f939c101de0 0x7f939c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 >> 172.21.15.31:0/1875206638 conn(0x7f939c0fd8c0 msgr2=0x7f939c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 shutdown_connections 2024-09-16T00:38:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:30.380+0000 7f93a45b4640 1 -- 172.21.15.31:0/1875206638 wait complete. 2024-09-16T00:38:30.693 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:30 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1875206638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:30.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:30 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1875206638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:30.827 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:30 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1875206638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:31.247 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:38:31 smithi081 systemd[1]: Started Ceph mds.h for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:31.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:31 smithi102 ceph-mon[35364]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:31.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:31.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:31.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:31.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:31.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:31.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:31 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:31.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:31 smithi031 ceph-mon[30956]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:31.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:31.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:31.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:31.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:31.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:31.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:31 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:31.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:31 smithi081 ceph-mon[35539]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:31.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:31.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:31.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:31.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:31.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:31.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:31 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:32.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:32 smithi102 ceph-mon[35364]: Deploying daemon mds.i on smithi102 2024-09-16T00:38:32.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:32 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.81:6836/1338256650,v1:172.21.15.81:6837/1338256650] up:boot 2024-09-16T00:38:32.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:32 smithi102 ceph-mon[35364]: fsmap 8 up:standby 2024-09-16T00:38:32.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:32 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:38:32.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:32 smithi031 ceph-mon[30956]: Deploying daemon mds.i on smithi102 2024-09-16T00:38:32.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:32 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.81:6836/1338256650,v1:172.21.15.81:6837/1338256650] up:boot 2024-09-16T00:38:32.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:32 smithi031 ceph-mon[30956]: fsmap 8 up:standby 2024-09-16T00:38:32.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:32 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:38:32.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:32 smithi081 ceph-mon[35539]: Deploying daemon mds.i on smithi102 2024-09-16T00:38:32.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:32 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.81:6836/1338256650,v1:172.21.15.81:6837/1338256650] up:boot 2024-09-16T00:38:32.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:32 smithi081 ceph-mon[35539]: fsmap 8 up:standby 2024-09-16T00:38:32.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:32 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:38:33.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:33.285 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:38:33 smithi102 systemd[1]: Starting Ceph mds.i for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:33.333 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:33.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:33 smithi102 ceph-mon[35364]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:33.550 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:38:33 smithi102 podman[65292]: 2024-09-16 00:38:33.18778063 +0000 UTC m=+0.015498830 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:33.550 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:38:33 smithi102 podman[65292]: 2024-09-16 00:38:33.324944087 +0000 UTC m=+0.152662270 container create 76b0ed73d828e6c29cc60354366772d4bf017aac4281287699ed3da340849e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T00:38:33.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:33 smithi031 ceph-mon[30956]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:33.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.816+0000 7f74c50f1640 1 -- 172.21.15.31:0/3893185869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 msgr2=0x7f74c010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.816+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3893185869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 0x7f74c010bb40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009900 tx=0x7f74a802a480 comp rx=0 tx=0).stop 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.816+0000 7f74c50f1640 1 -- 172.21.15.31:0/3893185869 shutdown_connections 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.816+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3893185869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c010c160 0x7f74c010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.816+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3893185869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 0x7f74c010bb40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.816+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3893185869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74c0102d30 0x7f74c0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.816+0000 7f74c50f1640 1 -- 172.21.15.31:0/3893185869 >> 172.21.15.31:0/3893185869 conn(0x7f74c00fc790 msgr2=0x7f74c00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.816+0000 7f74c50f1640 1 -- 172.21.15.31:0/3893185869 shutdown_connections 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.816+0000 7f74c50f1640 1 -- 172.21.15.31:0/3893185869 wait complete. 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.817+0000 7f74c50f1640 1 Processor -- start 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.817+0000 7f74c50f1640 1 -- start start 2024-09-16T00:38:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.817+0000 7f74c50f1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c0102d30 0x7f74c0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.817+0000 7f74c50f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 0x7f74c0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.817+0000 7f74c50f1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74c010c160 0x7f74c0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.817+0000 7f74c50f1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0111fb0 con 0x7f74c0103650 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.817+0000 7f74c50f1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0111e30 con 0x7f74c010c160 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.817+0000 7f74c50f1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0112130 con 0x7f74c0102d30 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74be59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 0x7f74c0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74bed9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c0102d30 0x7f74c0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74be59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 0x7f74c0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52380/0 (socket says 172.21.15.31:52380) 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74be59c640 1 -- 172.21.15.31:0/3005800178 learned_addr learned my addr 172.21.15.31:0/3005800178 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74bf59e640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74c010c160 0x7f74c0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74be59c640 1 -- 172.21.15.31:0/3005800178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c0102d30 msgr2=0x7f74c0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74be59c640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c0102d30 0x7f74c0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74be59c640 1 -- 172.21.15.31:0/3005800178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74c010c160 msgr2=0x7f74c0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74be59c640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74c010c160 0x7f74c0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74be59c640 1 -- 172.21.15.31:0/3005800178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c01a6930 con 0x7f74c0103650 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74bed9d640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c0102d30 0x7f74c0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.818+0000 7f74be59c640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 0x7f74c0100ed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74a8004770 tx=0x7f74a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.819+0000 7f74b7fff640 1 -- 172.21.15.31:0/3005800178 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a803f070 con 0x7f74c0103650 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.819+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c01a6bc0 con 0x7f74c0103650 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.819+0000 7f74b7fff640 1 -- 172.21.15.31:0/3005800178 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74a8031ca0 con 0x7f74c0103650 2024-09-16T00:38:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.819+0000 7f74b7fff640 1 -- 172.21.15.31:0/3005800178 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8035030 con 0x7f74c0103650 2024-09-16T00:38:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.819+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c01a70f0 con 0x7f74c0103650 2024-09-16T00:38:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.820+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c0103110 con 0x7f74c0103650 2024-09-16T00:38:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.822+0000 7f74b7fff640 1 -- 172.21.15.31:0/3005800178 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f74a8042070 con 0x7f74c0103650 2024-09-16T00:38:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.822+0000 7f74b7fff640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f74ac0a3240 0x7f74ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.823+0000 7f74b7fff640 1 -- 172.21.15.31:0/3005800178 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f74a80efaf0 con 0x7f74c0103650 2024-09-16T00:38:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.823+0000 7f74bed9d640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f74ac0a3240 0x7f74ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.823+0000 7f74bed9d640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f74ac0a3240 0x7f74ac0a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f74b0005d00 tx=0x7f74b000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:33.829+0000 7f74b7fff640 1 -- 172.21.15.31:0/3005800178 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74a80bc660 con 0x7f74c0103650 2024-09-16T00:38:33.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:33 smithi081 ceph-mon[35539]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:34.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.046+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74c01a72c0 con 0x7f74c0103650 2024-09-16T00:38:34.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.047+0000 7f74b7fff640 1 -- 172.21.15.31:0/3005800178 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7f74a80c0960 con 0x7f74c0103650 2024-09-16T00:38:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:34.047 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"btime":"2024-09-16T00:38:31:507630+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-16T00:38:34.048 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-09-16T00:38:34.049 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:38:33 smithi102 podman[65292]: 2024-09-16 00:38:33.669750693 +0000 UTC m=+0.497468875 container init 76b0ed73d828e6c29cc60354366772d4bf017aac4281287699ed3da340849e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T00:38:34.049 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:38:33 smithi102 podman[65292]: 2024-09-16 00:38:33.67269366 +0000 UTC m=+0.500411854 container start 76b0ed73d828e6c29cc60354366772d4bf017aac4281287699ed3da340849e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T00:38:34.049 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:38:33 smithi102 ceph-mds[65311]: -- 172.21.15.102:0/3881007405 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563bc888b860 con 0x563bc8868800 2024-09-16T00:38:34.050 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:38:33 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i[65307]: starting mds.i at 2024-09-16T00:38:34.050 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:38:33 smithi102 bash[65292]: 76b0ed73d828e6c29cc60354366772d4bf017aac4281287699ed3da340849e8a 2024-09-16T00:38:34.050 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:38:33 smithi102 systemd[1]: Started Ceph mds.i for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f74ac0a3240 msgr2=0x7f74ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f74ac0a3240 0x7f74ac0a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f74b0005d00 tx=0x7f74b000b410 comp rx=0 tx=0).stop 2024-09-16T00:38:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 msgr2=0x7f74c0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 0x7f74c0100ed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74a8004770 tx=0x7f74a8003c10 comp rx=0 tx=0).stop 2024-09-16T00:38:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 shutdown_connections 2024-09-16T00:38:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74c010c160 0x7f74c0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c0103650 0x7f74c0100ed0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f74ac0a3240 0x7f74ac0a5700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 --2- 172.21.15.31:0/3005800178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c0102d30 0x7f74c0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 >> 172.21.15.31:0/3005800178 conn(0x7f74c00fc790 msgr2=0x7f74c00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 shutdown_connections 2024-09-16T00:38:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:34.049+0000 7f74c50f1640 1 -- 172.21.15.31:0/3005800178 wait complete. 2024-09-16T00:38:35.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: Deploying daemon mds.g on smithi031 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3005800178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.102:6838/3021718993,v1:172.21.15.102:6839/3021718993] up:boot 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: fsmap 9 up:standby 2024-09-16T00:38:35.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:34 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:38:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:35.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:35.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: Deploying daemon mds.g on smithi031 2024-09-16T00:38:35.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3005800178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:35.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.102:6838/3021718993,v1:172.21.15.102:6839/3021718993] up:boot 2024-09-16T00:38:35.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: fsmap 9 up:standby 2024-09-16T00:38:35.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:34 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:38:35.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:35.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:35.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:35.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:35.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:35.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:35.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:35.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: Deploying daemon mds.g on smithi031 2024-09-16T00:38:35.301 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3005800178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:35.301 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.102:6838/3021718993,v1:172.21.15.102:6839/3021718993] up:boot 2024-09-16T00:38:35.301 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: fsmap 9 up:standby 2024-09-16T00:38:35.301 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:34 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:38:36.626 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:38:36 smithi031 systemd[1]: Starting Ceph mds.g for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:36.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:37.037 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:36 smithi031 ceph-mon[30956]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:37.037 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:38:36 smithi031 podman[69028]: 2024-09-16 00:38:36.656643267 +0000 UTC m=+0.016725965 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:37.037 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:38:36 smithi031 podman[69028]: 2024-09-16 00:38:36.775942157 +0000 UTC m=+0.136024833 container create c8e9d3c8e852688957255669f9b12c6203497a69f2565c3749edd582b2050475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T00:38:37.037 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:38:37 smithi031 podman[69028]: 2024-09-16 00:38:37.037199901 +0000 UTC m=+0.397282588 container init c8e9d3c8e852688957255669f9b12c6203497a69f2565c3749edd582b2050475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T00:38:37.240 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:37.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:36 smithi081 ceph-mon[35539]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:37.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:36 smithi102 ceph-mon[35364]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:37.363 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:38:37 smithi031 podman[69028]: 2024-09-16 00:38:37.040572564 +0000 UTC m=+0.400655248 container start c8e9d3c8e852688957255669f9b12c6203497a69f2565c3749edd582b2050475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-16T00:38:37.364 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:38:37 smithi031 ceph-mds[69077]: -- 172.21.15.31:0/1925417620 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7463c5860 con 0x55d7463a3400 2024-09-16T00:38:37.364 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:38:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-g[69061]: starting mds.g at 2024-09-16T00:38:37.364 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:38:37 smithi031 bash[69028]: c8e9d3c8e852688957255669f9b12c6203497a69f2565c3749edd582b2050475 2024-09-16T00:38:37.364 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:38:37 smithi031 systemd[1]: Started Ceph mds.g for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:37.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.749+0000 7f5dd2674640 1 -- 172.21.15.31:0/1523637235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc100e20 msgr2=0x7f5dcc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.749+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1523637235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc100e20 0x7f5dcc101200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009900 tx=0x7f5dbc02a480 comp rx=0 tx=0).stop 2024-09-16T00:38:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.749+0000 7f5dd2674640 1 -- 172.21.15.31:0/1523637235 shutdown_connections 2024-09-16T00:38:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.749+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1523637235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dcc102ef0 0x7f5dcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.749+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1523637235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc102570 0x7f5dcc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.749+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1523637235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc100e20 0x7f5dcc101200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.749+0000 7f5dd2674640 1 -- 172.21.15.31:0/1523637235 >> 172.21.15.31:0/1523637235 conn(0x7f5dcc0fc7d0 msgr2=0x7f5dcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.749+0000 7f5dd2674640 1 -- 172.21.15.31:0/1523637235 shutdown_connections 2024-09-16T00:38:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.749+0000 7f5dd2674640 1 -- 172.21.15.31:0/1523637235 wait complete. 2024-09-16T00:38:37.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd2674640 1 Processor -- start 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd2674640 1 -- start start 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd2674640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc100e20 0x7f5dcc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd2674640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dcc102570 0x7f5dcc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd2674640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc102ef0 0x7f5dcc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd2674640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc110f80 con 0x7f5dcc102ef0 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd2674640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc110e00 con 0x7f5dcc102570 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd2674640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc111100 con 0x7f5dcc100e20 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd0c11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc102ef0 0x7f5dcc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd0c11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc102ef0 0x7f5dcc19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35580/0 (socket says 172.21.15.31:35580) 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dd0c11640 1 -- 172.21.15.31:0/1130531449 learned_addr learned my addr 172.21.15.31:0/1130531449 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.750+0000 7f5dcbfff640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc100e20 0x7f5dcc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dcb7fe640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dcc102570 0x7f5dcc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dd0c11640 1 -- 172.21.15.31:0/1130531449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc100e20 msgr2=0x7f5dcc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dd0c11640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc100e20 0x7f5dcc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dd0c11640 1 -- 172.21.15.31:0/1130531449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dcc102570 msgr2=0x7f5dcc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dd0c11640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dcc102570 0x7f5dcc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dd0c11640 1 -- 172.21.15.31:0/1130531449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc19d4b0 con 0x7f5dcc102ef0 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dd0c11640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc102ef0 0x7f5dcc19cdd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400eaa0 tx=0x7f5dc400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dc97fa640 1 -- 172.21.15.31:0/1130531449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc400cdb0 con 0x7f5dcc102ef0 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dc97fa640 1 -- 172.21.15.31:0/1130531449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc4005020 con 0x7f5dcc102ef0 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc19d7a0 con 0x7f5dcc102ef0 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dc97fa640 1 -- 172.21.15.31:0/1130531449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40116b0 con 0x7f5dcc102ef0 2024-09-16T00:38:37.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.751+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc1a5080 con 0x7f5dcc102ef0 2024-09-16T00:38:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.752+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dcc101200 con 0x7f5dcc102ef0 2024-09-16T00:38:37.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.754+0000 7f5dc97fa640 1 -- 172.21.15.31:0/1130531449 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5dc4005560 con 0x7f5dcc102ef0 2024-09-16T00:38:37.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.755+0000 7f5dc97fa640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5db40a32c0 0x7f5db40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:37.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.755+0000 7f5dcbfff640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5db40a32c0 0x7f5db40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:37.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.755+0000 7f5dc97fa640 1 -- 172.21.15.31:0/1130531449 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f5dc40cf730 con 0x7f5dcc102ef0 2024-09-16T00:38:37.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.756+0000 7f5dcbfff640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5db40a32c0 0x7f5db40a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009230 tx=0x7f5dbc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:37.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.760+0000 7f5dc97fa640 1 -- 172.21.15.31:0/1130531449 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5dc4016030 con 0x7f5dcc102ef0 2024-09-16T00:38:37.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.983+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dcc1a5250 con 0x7f5dcc102ef0 2024-09-16T00:38:37.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.984+0000 7f5dc97fa640 1 -- 172.21.15.31:0/1130531449 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7f5dc409c850 con 0x7f5dcc102ef0 2024-09-16T00:38:37.984 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:37.985 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":11,"btime":"2024-09-16T00:38:37:823886+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T00:38:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 11 2024-09-16T00:38:37.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5db40a32c0 msgr2=0x7f5db40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5db40a32c0 0x7f5db40a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009230 tx=0x7f5dbc0058d0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc102ef0 msgr2=0x7f5dcc19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc102ef0 0x7f5dcc19cdd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400eaa0 tx=0x7f5dc400ef60 comp rx=0 tx=0).stop 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 shutdown_connections 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc102ef0 0x7f5dcc19cdd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dcc102570 0x7f5dcc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5db40a32c0 0x7f5db40a5780 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 --2- 172.21.15.31:0/1130531449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc100e20 0x7f5dcc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 >> 172.21.15.31:0/1130531449 conn(0x7f5dcc0fc7d0 msgr2=0x7f5dcc06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 shutdown_connections 2024-09-16T00:38:37.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:37.986+0000 7f5dd2674640 1 -- 172.21.15.31:0/1130531449 wait complete. 2024-09-16T00:38:38.322 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: Deploying daemon mds.a on smithi031 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.31:6838/3406901465,v1:172.21.15.31:6839/3406901465] up:boot 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: fsmap 10 up:standby 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:38:38.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:38 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1130531449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:38.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: Deploying daemon mds.a on smithi031 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.31:6838/3406901465,v1:172.21.15.31:6839/3406901465] up:boot 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: fsmap 10 up:standby 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:38:38.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:38 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1130531449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:38.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:38.628 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:38.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:38.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:38.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:38.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:38.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: Deploying daemon mds.a on smithi031 2024-09-16T00:38:38.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.31:6838/3406901465,v1:172.21.15.31:6839/3406901465] up:boot 2024-09-16T00:38:38.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: fsmap 10 up:standby 2024-09-16T00:38:38.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:38:38.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:38 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1130531449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:39.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:39 smithi081 ceph-mon[35539]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:39.522 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:39 smithi031 ceph-mon[30956]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:39.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:39 smithi102 ceph-mon[35364]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:40.125 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:38:40 smithi031 systemd[1]: Starting Ceph mds.a for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:40.488 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:38:40 smithi031 podman[69441]: 2024-09-16 00:38:40.133468994 +0000 UTC m=+0.016440842 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:40.489 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:38:40 smithi031 podman[69441]: 2024-09-16 00:38:40.23279613 +0000 UTC m=+0.115767968 container create ce1c6b7e983dac015b8458a3d042d838610475f2c10428b4e6bc4f53a4602caf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:38:40.489 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:38:40 smithi031 podman[69441]: 2024-09-16 00:38:40.460615101 +0000 UTC m=+0.343586937 container init ce1c6b7e983dac015b8458a3d042d838610475f2c10428b4e6bc4f53a4602caf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T00:38:40.489 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:38:40 smithi031 podman[69441]: 2024-09-16 00:38:40.463402625 +0000 UTC m=+0.346374460 container start ce1c6b7e983dac015b8458a3d042d838610475f2c10428b4e6bc4f53a4602caf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:38:40.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:40.758 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:38:40 smithi031 ceph-mds[69459]: -- 172.21.15.31:0/1718005145 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5602d9c91860 con 0x5602d9c6f400 2024-09-16T00:38:40.759 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:38:40 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-a[69455]: starting mds.a at 2024-09-16T00:38:40.759 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:38:40 smithi031 bash[69441]: ce1c6b7e983dac015b8458a3d042d838610475f2c10428b4e6bc4f53a4602caf 2024-09-16T00:38:40.759 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:38:40 smithi031 systemd[1]: Started Ceph mds.a for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:41.314 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:41 smithi081 ceph-mon[35539]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:41.314 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:41.314 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:41.314 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:41.314 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:41.314 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:41.314 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:41 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:41 smithi031 ceph-mon[30956]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:41.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:41.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:41.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:41.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:41.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:41 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:41.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.480+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/3765959156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8100a80 msgr2=0x7fb9e8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.480+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/3765959156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8100a80 0x7fb9e8100ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009930 tx=0x7fb9d802a420 comp rx=0 tx=0).stop 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.480+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/3765959156 shutdown_connections 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.480+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/3765959156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e8101420 0x7fb9e810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.480+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/3765959156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8100a80 0x7fb9e8100ee0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.480+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/3765959156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e8106a80 0x7fb9e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.480+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/3765959156 >> 172.21.15.31:0/3765959156 conn(0x7fb9e80fc7a0 msgr2=0x7fb9e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/3765959156 shutdown_connections 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/3765959156 wait complete. 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 Processor -- start 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 -- start start 2024-09-16T00:38:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e8100a80 0x7fb9e81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8101420 0x7fb9e81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e8106a80 0x7fb9e81a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8110e80 con 0x7fb9e8101420 2024-09-16T00:38:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8110d00 con 0x7fb9e8106a80 2024-09-16T00:38:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.481+0000 7fb9ef6ab640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8111000 con 0x7fb9e8100a80 2024-09-16T00:38:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ecc46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8101420 0x7fb9e81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ecc46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8101420 0x7fb9e81997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35616/0 (socket says 172.21.15.31:35616) 2024-09-16T00:38:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ecc46640 1 -- 172.21.15.31:0/806276034 learned_addr learned my addr 172.21.15.31:0/806276034 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ed447640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e8100a80 0x7fb9e81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9edc48640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e8106a80 0x7fb9e81a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ecc46640 1 -- 172.21.15.31:0/806276034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e8100a80 msgr2=0x7fb9e81992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ecc46640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e8100a80 0x7fb9e81992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ecc46640 1 -- 172.21.15.31:0/806276034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e8106a80 msgr2=0x7fb9e81a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ecc46640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e8106a80 0x7fb9e81a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ecc46640 1 -- 172.21.15.31:0/806276034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e81a4860 con 0x7fb9e8101420 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9edc48640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e8106a80 0x7fb9e81a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ecc46640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8101420 0x7fb9e81997e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009230 tx=0x7fb9d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9de7fc640 1 -- 172.21.15.31:0/806276034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d803f030 con 0x7fb9e8101420 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e81a4a30 con 0x7fb9e8101420 2024-09-16T00:38:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9de7fc640 1 -- 172.21.15.31:0/806276034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d8003ad0 con 0x7fb9e8101420 2024-09-16T00:38:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9de7fc640 1 -- 172.21.15.31:0/806276034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8035070 con 0x7fb9e8101420 2024-09-16T00:38:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.482+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e81a4ee0 con 0x7fb9e8101420 2024-09-16T00:38:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.483+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e8100ee0 con 0x7fb9e8101420 2024-09-16T00:38:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.485+0000 7fb9de7fc640 1 -- 172.21.15.31:0/806276034 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb9d8042030 con 0x7fb9e8101420 2024-09-16T00:38:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.485+0000 7fb9de7fc640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb9d40a3240 0x7fb9d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.485+0000 7fb9de7fc640 1 -- 172.21.15.31:0/806276034 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fb9d80efda0 con 0x7fb9e8101420 2024-09-16T00:38:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.485+0000 7fb9ed447640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb9d40a3240 0x7fb9d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.486+0000 7fb9ed447640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb9d40a3240 0x7fb9d40a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400bb60 tx=0x7fb9e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.489+0000 7fb9de7fc640 1 -- 172.21.15.31:0/806276034 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9d80bcf60 con 0x7fb9e8101420 2024-09-16T00:38:41.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:41 smithi102 ceph-mon[35364]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:41.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:41.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:41.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:41.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:38:41.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:38:41.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:41 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:41.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.701+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9e8109010 con 0x7fb9e8101420 2024-09-16T00:38:41.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.702+0000 7fb9de7fc640 1 -- 172.21.15.31:0/806276034 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7fb9d80bc860 con 0x7fb9e8101420 2024-09-16T00:38:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:41.702 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"btime":"2024-09-16T00:38:41:214693+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T00:38:41.703 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2024-09-16T00:38:41.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb9d40a3240 msgr2=0x7fb9d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:41.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb9d40a3240 0x7fb9d40a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400bb60 tx=0x7fb9e400b390 comp rx=0 tx=0).stop 2024-09-16T00:38:41.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8101420 msgr2=0x7fb9e81997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:41.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8101420 0x7fb9e81997e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009230 tx=0x7fb9d8004270 comp rx=0 tx=0).stop 2024-09-16T00:38:41.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 shutdown_connections 2024-09-16T00:38:41.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e8106a80 0x7fb9e81a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:41.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e8101420 0x7fb9e81997e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:41.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb9d40a3240 0x7fb9d40a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:41.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 --2- 172.21.15.31:0/806276034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e8100a80 0x7fb9e81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:41.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 >> 172.21.15.31:0/806276034 conn(0x7fb9e80fc7a0 msgr2=0x7fb9e80fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:41.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 shutdown_connections 2024-09-16T00:38:41.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:41.704+0000 7fb9ef6ab640 1 -- 172.21.15.31:0/806276034 wait complete. 2024-09-16T00:38:42.351 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:42 smithi081 ceph-mon[35539]: Deploying daemon mds.b on smithi081 2024-09-16T00:38:42.351 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:42 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.31:6840/3311366678,v1:172.21.15.31:6841/3311366678] up:boot 2024-09-16T00:38:42.351 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:42 smithi081 ceph-mon[35539]: fsmap 11 up:standby 2024-09-16T00:38:42.351 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:42 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:38:42.351 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:42 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/806276034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:42 smithi031 ceph-mon[30956]: Deploying daemon mds.b on smithi081 2024-09-16T00:38:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:42 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.31:6840/3311366678,v1:172.21.15.31:6841/3311366678] up:boot 2024-09-16T00:38:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:42 smithi031 ceph-mon[30956]: fsmap 11 up:standby 2024-09-16T00:38:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:42 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:38:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:42 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/806276034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:42.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:42 smithi102 ceph-mon[35364]: Deploying daemon mds.b on smithi081 2024-09-16T00:38:42.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:42 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.31:6840/3311366678,v1:172.21.15.31:6841/3311366678] up:boot 2024-09-16T00:38:42.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:42 smithi102 ceph-mon[35364]: fsmap 11 up:standby 2024-09-16T00:38:42.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:42 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:38:42.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:42 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/806276034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:42.747 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:42 smithi081 systemd[1]: Starting Ceph mds.b for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:38:43.030 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:42 smithi081 podman[65553]: 2024-09-16 00:38:42.654379912 +0000 UTC m=+0.015639508 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:38:43.030 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:42 smithi081 podman[65553]: 2024-09-16 00:38:42.793754037 +0000 UTC m=+0.155013672 container create 00e3bc8e521c12196171330c3e7cc1fc03235a1edf0b684cf9cf0694051eccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:38:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:43 smithi081 ceph-mon[35539]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:43 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:43.497 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:43 smithi081 podman[65553]: 2024-09-16 00:38:43.030031426 +0000 UTC m=+0.391291018 container init 00e3bc8e521c12196171330c3e7cc1fc03235a1edf0b684cf9cf0694051eccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T00:38:43.497 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:43 smithi081 podman[65553]: 2024-09-16 00:38:43.033628841 +0000 UTC m=+0.394888436 container start 00e3bc8e521c12196171330c3e7cc1fc03235a1edf0b684cf9cf0694051eccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T00:38:43.497 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:43 smithi081 ceph-mds[65571]: -- 172.21.15.81:0/2803314343 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ddb3b5860 con 0x560ddb392800 2024-09-16T00:38:43.498 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:43 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b[65567]: starting mds.b at 2024-09-16T00:38:43.498 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:43 smithi081 bash[65553]: 00e3bc8e521c12196171330c3e7cc1fc03235a1edf0b684cf9cf0694051eccfa 2024-09-16T00:38:43.498 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:43 smithi081 systemd[1]: Started Ceph mds.b for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:38:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:43 smithi102 ceph-mon[35364]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:43 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:43.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:43 smithi031 ceph-mon[30956]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:43.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:43 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:38:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:44 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] up:boot 2024-09-16T00:38:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:44 smithi081 ceph-mon[35539]: fsmap 12 up:standby 2024-09-16T00:38:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:38:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:44 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:44.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:44 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] up:boot 2024-09-16T00:38:44.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:44 smithi102 ceph-mon[35364]: fsmap 12 up:standby 2024-09-16T00:38:44.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:38:44.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:44 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:44.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:44 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] up:boot 2024-09-16T00:38:44.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:44 smithi031 ceph-mon[30956]: fsmap 12 up:standby 2024-09-16T00:38:44.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:38:44.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:44.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:44 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/876760504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4101420 msgr2=0x7f8bf410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/876760504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4101420 0x7f8bf410d950 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009cd0 tx=0x7f8bec02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/876760504 shutdown_connections 2024-09-16T00:38:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/876760504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4101420 0x7f8bf410d950 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/876760504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bf4100a80 0x7f8bf4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/876760504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf4106a80 0x7f8bf4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/876760504 >> 172.21.15.31:0/876760504 conn(0x7f8bf40fc7a0 msgr2=0x7f8bf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/876760504 shutdown_connections 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/876760504 wait complete. 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 Processor -- start 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.246+0000 7f8bfc1ae640 1 -- start start 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfc1ae640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bf4100a80 0x7f8bf40fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfc1ae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf4101420 0x7f8bf40fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfc1ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4106a80 0x7f8bf40fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfc1ae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4110e80 con 0x7f8bf4106a80 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfc1ae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4110d00 con 0x7f8bf4101420 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfc1ae640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4111000 con 0x7f8bf4100a80 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfa74b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4106a80 0x7f8bf40fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfa74b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4106a80 0x7f8bf40fd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35638/0 (socket says 172.21.15.31:35638) 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfa74b640 1 -- 172.21.15.31:0/1195604006 learned_addr learned my addr 172.21.15.31:0/1195604006 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bf9f4a640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bf4100a80 0x7f8bf40fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bf9749640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf4101420 0x7f8bf40fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfa74b640 1 -- 172.21.15.31:0/1195604006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bf4100a80 msgr2=0x7f8bf40fcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfa74b640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bf4100a80 0x7f8bf40fcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfa74b640 1 -- 172.21.15.31:0/1195604006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf4101420 msgr2=0x7f8bf40fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfa74b640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf4101420 0x7f8bf40fd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfa74b640 1 -- 172.21.15.31:0/1195604006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf41a6930 con 0x7f8bf4106a80 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.247+0000 7f8bfa74b640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4106a80 0x7f8bf40fd810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8bec005cc0 tx=0x7f8bec032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.251+0000 7f8beaffd640 1 -- 172.21.15.31:0/1195604006 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004040 con 0x7f8bf4106a80 2024-09-16T00:38:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.252+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf41a6bc0 con 0x7f8bf4106a80 2024-09-16T00:38:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.252+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a7150 con 0x7f8bf4106a80 2024-09-16T00:38:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.252+0000 7f8beaffd640 1 -- 172.21.15.31:0/1195604006 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bec0046c0 con 0x7f8bf4106a80 2024-09-16T00:38:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.252+0000 7f8beaffd640 1 -- 172.21.15.31:0/1195604006 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec030c00 con 0x7f8bf4106a80 2024-09-16T00:38:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.253+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf4100ee0 con 0x7f8bf4106a80 2024-09-16T00:38:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.254+0000 7f8beaffd640 1 -- 172.21.15.31:0/1195604006 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8bec0305a0 con 0x7f8bf4106a80 2024-09-16T00:38:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.254+0000 7f8beaffd640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8be00a3240 0x7f8be00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.254+0000 7f8bf9f4a640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8be00a3240 0x7f8be00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.254+0000 7f8beaffd640 1 -- 172.21.15.31:0/1195604006 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f8bec0f09e0 con 0x7f8bf4106a80 2024-09-16T00:38:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.255+0000 7f8bf9f4a640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8be00a3240 0x7f8be00a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009b50 tx=0x7f8bf0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.258+0000 7f8beaffd640 1 -- 172.21.15.31:0/1195604006 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8bec04b060 con 0x7f8bf4106a80 2024-09-16T00:38:45.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.480+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bf410a000 con 0x7f8bf4106a80 2024-09-16T00:38:45.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:45 smithi031 ceph-mon[30956]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:45.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:45 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.481+0000 7f8beaffd640 1 -- 172.21.15.31:0/1195604006 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7f8bec0bdba0 con 0x7f8bf4106a80 2024-09-16T00:38:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":13,"btime":"2024-09-16T00:38:43:227327+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-16T00:38:45.482 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 13 2024-09-16T00:38:45.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8be00a3240 msgr2=0x7f8be00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8be00a3240 0x7f8be00a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009b50 tx=0x7f8bf0005f90 comp rx=0 tx=0).stop 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4106a80 msgr2=0x7f8bf40fd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4106a80 0x7f8bf40fd810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8bec005cc0 tx=0x7f8bec032f40 comp rx=0 tx=0).stop 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 shutdown_connections 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf4106a80 0x7f8bf40fd810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf4101420 0x7f8bf40fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8be00a3240 0x7f8be00a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 --2- 172.21.15.31:0/1195604006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bf4100a80 0x7f8bf40fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 >> 172.21.15.31:0/1195604006 conn(0x7f8bf40fc7a0 msgr2=0x7f8bf406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 shutdown_connections 2024-09-16T00:38:45.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:45.483+0000 7f8bfc1ae640 1 -- 172.21.15.31:0/1195604006 wait complete. 2024-09-16T00:38:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:45 smithi102 ceph-mon[35364]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:45 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:45 smithi081 ceph-mon[35539]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:45 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-16T00:38:46.339 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.339 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.339 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:46.339 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:46.339 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.339 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:46 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:46.339 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:46 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1195604006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:46.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:46 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:46.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:46 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1195604006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:46.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:46.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:46.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:46.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:46 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:38:46.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:46 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1195604006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:38:47.066 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:47.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:47 smithi031 ceph-mon[30956]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:47.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:47 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 -- 172.21.15.31:0/118661447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 msgr2=0x7fca2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 --2- 172.21.15.31:0/118661447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 0x7fca2410ea90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009ca0 tx=0x7fca1c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 -- 172.21.15.31:0/118661447 shutdown_connections 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 --2- 172.21.15.31:0/118661447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 0x7fca2410ea90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 --2- 172.21.15.31:0/118661447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca24102790 0x7fca241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 --2- 172.21.15.31:0/118661447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca24101de0 0x7fca241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 -- 172.21.15.31:0/118661447 >> 172.21.15.31:0/118661447 conn(0x7fca240fd8c0 msgr2=0x7fca240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 -- 172.21.15.31:0/118661447 shutdown_connections 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 -- 172.21.15.31:0/118661447 wait complete. 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 Processor -- start 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.522+0000 7fca2ab79640 1 -- start start 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca2ab79640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca24101de0 0x7fca240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca2ab79640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca24102790 0x7fca24074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca2ab79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 0x7fca24074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca2ab79640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fca24111f90 con 0x7fca24106f10 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca2ab79640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fca24111e10 con 0x7fca24101de0 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca2ab79640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fca24112110 con 0x7fca24102790 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca29116640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 0x7fca24074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca29116640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 0x7fca24074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59550/0 (socket says 172.21.15.31:59550) 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca29116640 1 -- 172.21.15.31:0/3525490306 learned_addr learned my addr 172.21.15.31:0/3525490306 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca1bfff640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca24102790 0x7fca24074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca28915640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca24101de0 0x7fca240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca29116640 1 -- 172.21.15.31:0/3525490306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca24102790 msgr2=0x7fca24074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca29116640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca24102790 0x7fca24074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca29116640 1 -- 172.21.15.31:0/3525490306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca24101de0 msgr2=0x7fca240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca29116640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca24101de0 0x7fca240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca29116640 1 -- 172.21.15.31:0/3525490306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca241a6930 con 0x7fca24106f10 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca29116640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 0x7fca24074f70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca1c031a20 tx=0x7fca1c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca19ffb640 1 -- 172.21.15.31:0/3525490306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c003c30 con 0x7fca24106f10 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.523+0000 7fca19ffb640 1 -- 172.21.15.31:0/3525490306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1c003dd0 con 0x7fca24106f10 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.524+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca241a6bc0 con 0x7fca24106f10 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.524+0000 7fca19ffb640 1 -- 172.21.15.31:0/3525490306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c02fcf0 con 0x7fca24106f10 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.524+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca241a7150 con 0x7fca24106f10 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.525+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca241021c0 con 0x7fca24106f10 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.526+0000 7fca19ffb640 1 -- 172.21.15.31:0/3525490306 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fca1c004b40 con 0x7fca24106f10 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.527+0000 7fca19ffb640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fca0c0a32c0 0x7fca0c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.527+0000 7fca28915640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fca0c0a32c0 0x7fca0c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.527+0000 7fca19ffb640 1 -- 172.21.15.31:0/3525490306 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fca1c0f09e0 con 0x7fca24106f10 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.527+0000 7fca28915640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fca0c0a32c0 0x7fca0c0a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fca20005fd0 tx=0x7fca20005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.530+0000 7fca19ffb640 1 -- 172.21.15.31:0/3525490306 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca1c0bdb90 con 0x7fca24106f10 2024-09-16T00:38:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.720+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fca24075a20 con 0x7fca24106f10 2024-09-16T00:38:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.720+0000 7fca19ffb640 1 -- 172.21.15.31:0/3525490306 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7fca24075a20 con 0x7fca24106f10 2024-09-16T00:38:47.721 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:38:47.722 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-09-16T00:38:47.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:38:47.722 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-09-16T00:38:47.722 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-09-16T00:38:47.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:38:47.722 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-09-16T00:38:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.722+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fca0c0a32c0 msgr2=0x7fca0c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.722+0000 7fca2ab79640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fca0c0a32c0 0x7fca0c0a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fca20005fd0 tx=0x7fca20005850 comp rx=0 tx=0).stop 2024-09-16T00:38:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.722+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 msgr2=0x7fca24074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.722+0000 7fca2ab79640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 0x7fca24074f70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca1c031a20 tx=0x7fca1c031a50 comp rx=0 tx=0).stop 2024-09-16T00:38:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.723+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 shutdown_connections 2024-09-16T00:38:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.723+0000 7fca2ab79640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca24106f10 0x7fca24074f70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.723+0000 7fca2ab79640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fca0c0a32c0 0x7fca0c0a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.723+0000 7fca2ab79640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca24102790 0x7fca24074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.723+0000 7fca2ab79640 1 --2- 172.21.15.31:0/3525490306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca24101de0 0x7fca240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:47.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.723+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 >> 172.21.15.31:0/3525490306 conn(0x7fca240fd8c0 msgr2=0x7fca240fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:47.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.723+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 shutdown_connections 2024-09-16T00:38:47.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:47.723+0000 7fca2ab79640 1 -- 172.21.15.31:0/3525490306 wait complete. 2024-09-16T00:38:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:47 smithi081 ceph-mon[35539]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:47 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:47 smithi102 ceph-mon[35364]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:47.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:47.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:47 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:48.443 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T00:38:48.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T00:38:48.711 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:48.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:48.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:48 smithi031 ceph-mon[30956]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:48.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:48 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3525490306' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:38:48.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:48.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:48.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:48.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:48 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:48 smithi081 ceph-mon[35539]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:48 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3525490306' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:38:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:48.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:48 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:49.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:49.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:48 smithi102 ceph-mon[35364]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:49.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:48 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3525490306' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:38:49.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:49.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:49.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:38:49.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:38:49.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:48 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:38:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.184+0000 7fdd086ce640 1 -- 172.21.15.31:0/880021038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00106f10 msgr2=0x7fdd0010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.184+0000 7fdd086ce640 1 --2- 172.21.15.31:0/880021038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00106f10 0x7fdd0010ea90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009ca0 tx=0x7fdcf802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 -- 172.21.15.31:0/880021038 shutdown_connections 2024-09-16T00:38:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 --2- 172.21.15.31:0/880021038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00106f10 0x7fdd0010ea90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 --2- 172.21.15.31:0/880021038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd00102790 0x7fdd001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 --2- 172.21.15.31:0/880021038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd00101de0 0x7fdd001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 -- 172.21.15.31:0/880021038 >> 172.21.15.31:0/880021038 conn(0x7fdd000fd8c0 msgr2=0x7fdd000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 -- 172.21.15.31:0/880021038 shutdown_connections 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 -- 172.21.15.31:0/880021038 wait complete. 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 Processor -- start 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 -- start start 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd00101de0 0x7fdd001985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.185+0000 7fdd086ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00102790 0x7fdd00198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd086ce640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd00106f10 0x7fdd0019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd086ce640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00112010 con 0x7fdd00102790 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd086ce640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00111e90 con 0x7fdd00106f10 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd086ce640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00112190 con 0x7fdd00101de0 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd05c69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00102790 0x7fdd00198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd05c69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00102790 0x7fdd00198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59556/0 (socket says 172.21.15.31:59556) 2024-09-16T00:38:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd05c69640 1 -- 172.21.15.31:0/1763316591 learned_addr learned my addr 172.21.15.31:0/1763316591 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd0646a640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd00101de0 0x7fdd001985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd06c6b640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd00106f10 0x7fdd0019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd05c69640 1 -- 172.21.15.31:0/1763316591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd00101de0 msgr2=0x7fdd001985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd05c69640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd00101de0 0x7fdd001985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd05c69640 1 -- 172.21.15.31:0/1763316591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd00106f10 msgr2=0x7fdd0019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd05c69640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd00106f10 0x7fdd0019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd05c69640 1 -- 172.21.15.31:0/1763316591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0019d5a0 con 0x7fdd00102790 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd06c6b640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd00106f10 0x7fdd0019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd05c69640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00102790 0x7fdd00198b30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000b4a0 tx=0x7fdcf000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdcf77fe640 1 -- 172.21.15.31:0/1763316591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0015020 con 0x7fdd00102790 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdcf77fe640 1 -- 172.21.15.31:0/1763316591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcf0011d10 con 0x7fdd00102790 2024-09-16T00:38:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0019d890 con 0x7fdd00102790 2024-09-16T00:38:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdcf77fe640 1 -- 172.21.15.31:0/1763316591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000f2e0 con 0x7fdd00102790 2024-09-16T00:38:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.186+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd001a5170 con 0x7fdd00102790 2024-09-16T00:38:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.187+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd00102240 con 0x7fdd00102790 2024-09-16T00:38:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.189+0000 7fdcf77fe640 1 -- 172.21.15.31:0/1763316591 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fdcf0018020 con 0x7fdd00102790 2024-09-16T00:38:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.189+0000 7fdcf77fe640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcec0a32c0 0x7fdcec0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.189+0000 7fdcf77fe640 1 -- 172.21.15.31:0/1763316591 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fdcf00cf100 con 0x7fdd00102790 2024-09-16T00:38:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.189+0000 7fdd0646a640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcec0a32c0 0x7fdcec0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.190+0000 7fdd0646a640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcec0a32c0 0x7fdcec0a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009990 tx=0x7fdcfc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.192+0000 7fdcf77fe640 1 -- 172.21.15.31:0/1763316591 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcf001f020 con 0x7fdd00102790 2024-09-16T00:38:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.368+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdd00199950 con 0x7fdd00102790 2024-09-16T00:38:49.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.857+0000 7fdcf77fe640 1 -- 172.21.15.31:0/1763316591 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7fdd00199950 con 0x7fdd00102790 2024-09-16T00:38:49.858 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-09-16T00:38:49.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcec0a32c0 msgr2=0x7fdcec0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcec0a32c0 0x7fdcec0a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009990 tx=0x7fdcfc009110 comp rx=0 tx=0).stop 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00102790 msgr2=0x7fdd00198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00102790 0x7fdd00198b30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000b4a0 tx=0x7fdcf000b8a0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 shutdown_connections 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd00106f10 0x7fdd0019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd00102790 0x7fdd00198b30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcec0a32c0 0x7fdcec0a5780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 --2- 172.21.15.31:0/1763316591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd00101de0 0x7fdd001985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 >> 172.21.15.31:0/1763316591 conn(0x7fdd000fd8c0 msgr2=0x7fdd000fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 shutdown_connections 2024-09-16T00:38:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:49.859+0000 7fdd086ce640 1 -- 172.21.15.31:0/1763316591 wait complete. 2024-09-16T00:38:50.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:49 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1763316591' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:38:50.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:49 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1763316591' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:38:50.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:49 smithi031 ceph-mon[30956]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:38:50.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:49 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1763316591' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:38:50.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:49 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1763316591' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:38:50.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:49 smithi081 ceph-mon[35539]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:38:50.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:49 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1763316591' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:38:50.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:49 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1763316591' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:38:50.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:49 smithi102 ceph-mon[35364]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:38:50.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-16T00:38:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:50.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:50 smithi031 ceph-mon[30956]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:50.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:50 smithi031 ceph-mon[30956]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:38:51.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:50 smithi081 ceph-mon[35539]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:51.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:50 smithi081 ceph-mon[35539]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:38:51.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:50 smithi102 ceph-mon[35364]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:51.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:50 smithi102 ceph-mon[35364]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:38:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.321+0000 7f5ebd41a640 1 -- 172.21.15.31:0/240183179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb8102d40 msgr2=0x7f5eb8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.321+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/240183179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb8102d40 0x7f5eb8103120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009900 tx=0x7f5ea802a480 comp rx=0 tx=0).stop 2024-09-16T00:38:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.321+0000 7f5ebd41a640 1 -- 172.21.15.31:0/240183179 shutdown_connections 2024-09-16T00:38:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.321+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/240183179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eb810c170 0x7f5eb810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.321+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/240183179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8103660 0x7f5eb810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.321+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/240183179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb8102d40 0x7f5eb8103120 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.321+0000 7f5ebd41a640 1 -- 172.21.15.31:0/240183179 >> 172.21.15.31:0/240183179 conn(0x7f5eb80fc7a0 msgr2=0x7f5eb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.321+0000 7f5ebd41a640 1 -- 172.21.15.31:0/240183179 shutdown_connections 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.321+0000 7f5ebd41a640 1 -- 172.21.15.31:0/240183179 wait complete. 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5ebd41a640 1 Processor -- start 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5ebd41a640 1 -- start start 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5ebd41a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eb8102d40 0x7f5eb8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5ebd41a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8103660 0x7f5eb8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5ebd41a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb810c170 0x7f5eb819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5ebd41a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb8111f70 con 0x7f5eb810c170 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5ebd41a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb8111df0 con 0x7f5eb8103660 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5ebd41a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81120f0 con 0x7f5eb8102d40 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5eb77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb810c170 0x7f5eb819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5eb77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb810c170 0x7f5eb819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59576/0 (socket says 172.21.15.31:59576) 2024-09-16T00:38:51.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5eb77fe640 1 -- 172.21.15.31:0/3914279547 learned_addr learned my addr 172.21.15.31:0/3914279547 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.322+0000 7f5eb67fc640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8103660 0x7f5eb8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5eb6ffd640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eb8102d40 0x7f5eb8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5eb77fe640 1 -- 172.21.15.31:0/3914279547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eb8102d40 msgr2=0x7f5eb8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5eb77fe640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eb8102d40 0x7f5eb8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5eb77fe640 1 -- 172.21.15.31:0/3914279547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8103660 msgr2=0x7f5eb8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5eb77fe640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8103660 0x7f5eb8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5eb77fe640 1 -- 172.21.15.31:0/3914279547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb819d400 con 0x7f5eb810c170 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5eb6ffd640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eb8102d40 0x7f5eb8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5eb77fe640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb810c170 0x7f5eb819cd20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000cce0 tx=0x7f5eb000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5e9ffff640 1 -- 172.21.15.31:0/3914279547 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000f450 con 0x7f5eb810c170 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5e9ffff640 1 -- 172.21.15.31:0/3914279547 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb000fad0 con 0x7f5eb810c170 2024-09-16T00:38:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb819d6f0 con 0x7f5eb810c170 2024-09-16T00:38:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5e9ffff640 1 -- 172.21.15.31:0/3914279547 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0018c10 con 0x7f5eb810c170 2024-09-16T00:38:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.323+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb81a4fd0 con 0x7f5eb810c170 2024-09-16T00:38:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.324+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb8103120 con 0x7f5eb810c170 2024-09-16T00:38:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.325+0000 7f5e9ffff640 1 -- 172.21.15.31:0/3914279547 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5eb000f5f0 con 0x7f5eb810c170 2024-09-16T00:38:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.326+0000 7f5e9ffff640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5ea00a3240 0x7f5ea00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.326+0000 7f5e9ffff640 1 -- 172.21.15.31:0/3914279547 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7f5eb00cfc60 con 0x7f5eb810c170 2024-09-16T00:38:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.326+0000 7f5eb6ffd640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5ea00a3240 0x7f5ea00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.326+0000 7f5eb6ffd640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5ea00a3240 0x7f5ea00a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8000c00 tx=0x7f5ea8032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.330+0000 7f5e9ffff640 1 -- 172.21.15.31:0/3914279547 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5eb0014030 con 0x7f5eb810c170 2024-09-16T00:38:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.509+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f5eb802d7c0 con 0x7f5eb810c170 2024-09-16T00:38:51.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.945+0000 7f5e9ffff640 1 -- 172.21.15.31:0/3914279547 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v79) ==== 171+0+0 (secure 0 0 0) 0x7f5eb009ccb0 con 0x7f5eb810c170 2024-09-16T00:38:51.947 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-09-16T00:38:51.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5ea00a3240 msgr2=0x7f5ea00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:51.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5ea00a3240 0x7f5ea00a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8000c00 tx=0x7f5ea8032c70 comp rx=0 tx=0).stop 2024-09-16T00:38:51.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb810c170 msgr2=0x7f5eb819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:51.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb810c170 0x7f5eb819cd20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000cce0 tx=0x7f5eb000a620 comp rx=0 tx=0).stop 2024-09-16T00:38:51.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 shutdown_connections 2024-09-16T00:38:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5eb810c170 0x7f5eb819cd20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8103660 0x7f5eb8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f5ea00a3240 0x7f5ea00a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 --2- 172.21.15.31:0/3914279547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eb8102d40 0x7f5eb8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 >> 172.21.15.31:0/3914279547 conn(0x7f5eb80fc7a0 msgr2=0x7f5eb80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 shutdown_connections 2024-09-16T00:38:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:51.948+0000 7f5ebd41a640 1 -- 172.21.15.31:0/3914279547 wait complete. 2024-09-16T00:38:52.278 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:51 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3914279547' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:38:52.279 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:51 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3914279547' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:38:52.279 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:51 smithi031 ceph-mon[30956]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:38:52.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:51 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3914279547' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:38:52.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:51 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3914279547' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:38:52.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:51 smithi102 ceph-mon[35364]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:38:52.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:51 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3914279547' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:38:52.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:51 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3914279547' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:38:52.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:51 smithi081 ceph-mon[35539]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:38:52.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T00:38:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:53.267 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:52 smithi031 ceph-mon[30956]: pgmap v275: 33 pgs: 6 creating+peering, 23 unknown, 4 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:53.267 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:52 smithi031 ceph-mon[30956]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:38:53.267 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:52 smithi031 ceph-mon[30956]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:38:53.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:52 smithi102 ceph-mon[35364]: pgmap v275: 33 pgs: 6 creating+peering, 23 unknown, 4 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:53.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:52 smithi102 ceph-mon[35364]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:38:53.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:52 smithi102 ceph-mon[35364]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:38:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.351+0000 7f1a893d6640 1 -- 172.21.15.31:0/2367298971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84102d30 msgr2=0x7f1a84103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.351+0000 7f1a893d6640 1 --2- 172.21.15.31:0/2367298971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84102d30 0x7f1a84103110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009930 tx=0x7f1a7402a420 comp rx=0 tx=0).stop 2024-09-16T00:38:53.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.351+0000 7f1a893d6640 1 -- 172.21.15.31:0/2367298971 shutdown_connections 2024-09-16T00:38:53.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.351+0000 7f1a893d6640 1 --2- 172.21.15.31:0/2367298971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a841082a0 0x7f1a8410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.351+0000 7f1a893d6640 1 --2- 172.21.15.31:0/2367298971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a841036e0 0x7f1a84107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.351+0000 7f1a893d6640 1 --2- 172.21.15.31:0/2367298971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84102d30 0x7f1a84103110 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.351+0000 7f1a893d6640 1 -- 172.21.15.31:0/2367298971 >> 172.21.15.31:0/2367298971 conn(0x7f1a840fc7d0 msgr2=0x7f1a840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.352+0000 7f1a893d6640 1 -- 172.21.15.31:0/2367298971 shutdown_connections 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.352+0000 7f1a893d6640 1 -- 172.21.15.31:0/2367298971 wait complete. 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.352+0000 7f1a893d6640 1 Processor -- start 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.352+0000 7f1a893d6640 1 -- start start 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.352+0000 7f1a893d6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a84102d30 0x7f1a841985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.352+0000 7f1a893d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a841036e0 0x7f1a84198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.352+0000 7f1a893d6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a841082a0 0x7f1a8419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a893d6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1a841120a0 con 0x7f1a841036e0 2024-09-16T00:38:53.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a893d6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1a84111f20 con 0x7f1a84102d30 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a893d6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1a84112220 con 0x7f1a841082a0 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a827fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a841036e0 0x7f1a84198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a827fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a841036e0 0x7f1a84198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59604/0 (socket says 172.21.15.31:59604) 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a827fc640 1 -- 172.21.15.31:0/436400851 learned_addr learned my addr 172.21.15.31:0/436400851 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a82ffd640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a84102d30 0x7f1a841985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a837fe640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a841082a0 0x7f1a8419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a827fc640 1 -- 172.21.15.31:0/436400851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a841082a0 msgr2=0x7f1a8419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a827fc640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a841082a0 0x7f1a8419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a827fc640 1 -- 172.21.15.31:0/436400851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a84102d30 msgr2=0x7f1a841985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a827fc640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a84102d30 0x7f1a841985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a827fc640 1 -- 172.21.15.31:0/436400851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8419d590 con 0x7f1a841036e0 2024-09-16T00:38:53.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a827fc640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a841036e0 0x7f1a84198b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800d750 tx=0x7f1a7800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a6bfff640 1 -- 172.21.15.31:0/436400851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78017070 con 0x7f1a841036e0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8419d880 con 0x7f1a841036e0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a6bfff640 1 -- 172.21.15.31:0/436400851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a78005500 con 0x7f1a841036e0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a6bfff640 1 -- 172.21.15.31:0/436400851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78004e00 con 0x7f1a841036e0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.353+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a841a5160 con 0x7f1a841036e0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.354+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8406a850 con 0x7f1a841036e0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.356+0000 7f1a6bfff640 1 -- 172.21.15.31:0/436400851 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1a78004fa0 con 0x7f1a841036e0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.356+0000 7f1a6bfff640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1a6c0a3240 0x7f1a6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.357+0000 7f1a6bfff640 1 -- 172.21.15.31:0/436400851 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f1a780cf4e0 con 0x7f1a841036e0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.357+0000 7f1a82ffd640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1a6c0a3240 0x7f1a6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.357+0000 7f1a82ffd640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1a6c0a3240 0x7f1a6c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009900 tx=0x7f1a740058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:53.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.360+0000 7f1a6bfff640 1 -- 172.21.15.31:0/436400851 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a7809c3b0 con 0x7f1a841036e0 2024-09-16T00:38:53.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:52 smithi081 ceph-mon[35539]: pgmap v275: 33 pgs: 6 creating+peering, 23 unknown, 4 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:53.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:52 smithi081 ceph-mon[35539]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:38:53.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:52 smithi081 ceph-mon[35539]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:38:53.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:53.574+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1a8410a2b0 con 0x7f1a841036e0 2024-09-16T00:38:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.004+0000 7f1a6bfff640 1 -- 172.21.15.31:0/436400851 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f1a7809bcb0 con 0x7f1a841036e0 2024-09-16T00:38:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T00:38:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1a6c0a3240 msgr2=0x7f1a6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1a6c0a3240 0x7f1a6c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009900 tx=0x7f1a740058d0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a841036e0 msgr2=0x7f1a84198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a841036e0 0x7f1a84198b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800d750 tx=0x7f1a7800dc10 comp rx=0 tx=0).stop 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 shutdown_connections 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1a6c0a3240 0x7f1a6c0a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a841082a0 0x7f1a8419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a841036e0 0x7f1a84198b20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 --2- 172.21.15.31:0/436400851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a84102d30 0x7f1a841985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.006+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 >> 172.21.15.31:0/436400851 conn(0x7f1a840fc7d0 msgr2=0x7f1a84105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.007+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 shutdown_connections 2024-09-16T00:38:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:54.007+0000 7f1a893d6640 1 -- 172.21.15.31:0/436400851 wait complete. 2024-09-16T00:38:54.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:54 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/436400851' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:38:54.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:54 smithi102 ceph-mon[35364]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:38:54.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:54 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/436400851' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:38:54.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:54 smithi031 ceph-mon[30956]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:38:54.466 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:54 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/436400851' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:38:54.467 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:54 smithi081 ceph-mon[35539]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:38:54.686 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-16T00:38:54.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-16T00:38:54.951 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:55.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:55 smithi102 ceph-mon[35364]: pgmap v278: 97 pgs: 19 creating+peering, 42 unknown, 36 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:55.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:55 smithi102 ceph-mon[35364]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:38:55.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:55 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/436400851' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:38:55.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:55 smithi102 ceph-mon[35364]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:38:55.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:55 smithi102 ceph-mon[35364]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:38:55.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:55 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] up:active 2024-09-16T00:38:55.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:55 smithi102 ceph-mon[35364]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:38:55.359 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:55 smithi031 ceph-mon[30956]: pgmap v278: 97 pgs: 19 creating+peering, 42 unknown, 36 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:55.359 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:55 smithi031 ceph-mon[30956]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:38:55.359 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:55 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/436400851' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:38:55.359 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:55 smithi031 ceph-mon[30956]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:38:55.359 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:55 smithi031 ceph-mon[30956]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:38:55.359 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:55 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] up:active 2024-09-16T00:38:55.359 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:55 smithi031 ceph-mon[30956]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:38:55.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.445+0000 7fc923aa2640 1 -- 172.21.15.31:0/2203204260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c106a80 msgr2=0x7fc91c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.445+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2203204260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c106a80 0x7fc91c106e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc918009930 tx=0x7fc91802a420 comp rx=0 tx=0).stop 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.445+0000 7fc923aa2640 1 -- 172.21.15.31:0/2203204260 shutdown_connections 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.445+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2203204260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc91c101420 0x7fc91c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.445+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2203204260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91c100a80 0x7fc91c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.445+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2203204260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c106a80 0x7fc91c106e60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.445+0000 7fc923aa2640 1 -- 172.21.15.31:0/2203204260 >> 172.21.15.31:0/2203204260 conn(0x7fc91c0fc7a0 msgr2=0x7fc91c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.445+0000 7fc923aa2640 1 -- 172.21.15.31:0/2203204260 shutdown_connections 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.445+0000 7fc923aa2640 1 -- 172.21.15.31:0/2203204260 wait complete. 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc923aa2640 1 Processor -- start 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc923aa2640 1 -- start start 2024-09-16T00:38:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc923aa2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91c100a80 0x7fc91c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc923aa2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c101420 0x7fc91c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc923aa2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc91c106a80 0x7fc91c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc923aa2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c111fa0 con 0x7fc91c101420 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc923aa2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c111e20 con 0x7fc91c100a80 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc923aa2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c112120 con 0x7fc91c106a80 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc92103d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c101420 0x7fc91c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc92103d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c101420 0x7fc91c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59626/0 (socket says 172.21.15.31:59626) 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc92103d640 1 -- 172.21.15.31:0/2491715074 learned_addr learned my addr 172.21.15.31:0/2491715074 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.446+0000 7fc92183e640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91c100a80 0x7fc91c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc92103d640 1 -- 172.21.15.31:0/2491715074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc91c106a80 msgr2=0x7fc91c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc92203f640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc91c106a80 0x7fc91c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc92103d640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc91c106a80 0x7fc91c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc92103d640 1 -- 172.21.15.31:0/2491715074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91c100a80 msgr2=0x7fc91c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc92103d640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91c100a80 0x7fc91c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc92103d640 1 -- 172.21.15.31:0/2491715074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91c19d450 con 0x7fc91c101420 2024-09-16T00:38:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc92203f640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc91c106a80 0x7fc91c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc92103d640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c101420 0x7fc91c1989e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc90c00d950 tx=0x7fc90c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc912ffd640 1 -- 172.21.15.31:0/2491715074 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c017030 con 0x7fc91c101420 2024-09-16T00:38:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91c19d740 con 0x7fc91c101420 2024-09-16T00:38:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc912ffd640 1 -- 172.21.15.31:0/2491715074 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc90c004020 con 0x7fc91c101420 2024-09-16T00:38:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc912ffd640 1 -- 172.21.15.31:0/2491715074 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0047a0 con 0x7fc91c101420 2024-09-16T00:38:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.447+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc91c1a5020 con 0x7fc91c101420 2024-09-16T00:38:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.448+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc91c100ee0 con 0x7fc91c101420 2024-09-16T00:38:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.449+0000 7fc912ffd640 1 -- 172.21.15.31:0/2491715074 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc90c0041d0 con 0x7fc91c101420 2024-09-16T00:38:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.450+0000 7fc912ffd640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc9080a3240 0x7fc9080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.450+0000 7fc92183e640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc9080a3240 0x7fc9080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.450+0000 7fc912ffd640 1 -- 172.21.15.31:0/2491715074 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fc90c0cf840 con 0x7fc91c101420 2024-09-16T00:38:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.451+0000 7fc92183e640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc9080a3240 0x7fc9080a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc918009900 tx=0x7fc9180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.454+0000 7fc912ffd640 1 -- 172.21.15.31:0/2491715074 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc90c09c6b0 con 0x7fc91c101420 2024-09-16T00:38:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:55 smithi081 ceph-mon[35539]: pgmap v278: 97 pgs: 19 creating+peering, 42 unknown, 36 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:38:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:55 smithi081 ceph-mon[35539]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:38:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:55 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/436400851' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:38:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:55 smithi081 ceph-mon[35539]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:38:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:55 smithi081 ceph-mon[35539]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:38:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:55 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] up:active 2024-09-16T00:38:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:55 smithi081 ceph-mon[35539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:38:55.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:55.638+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fc91c199800 con 0x7fc91c101420 2024-09-16T00:38:56.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.007+0000 7fc912ffd640 1 -- 172.21.15.31:0/2491715074 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v82) ==== 157+0+0 (secure 0 0 0) 0x7fc91c199800 con 0x7fc91c101420 2024-09-16T00:38:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc9080a3240 msgr2=0x7fc9080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc9080a3240 0x7fc9080a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc918009900 tx=0x7fc9180058d0 comp rx=0 tx=0).stop 2024-09-16T00:38:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c101420 msgr2=0x7fc91c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c101420 0x7fc91c1989e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc90c00d950 tx=0x7fc90c00de10 comp rx=0 tx=0).stop 2024-09-16T00:38:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 shutdown_connections 2024-09-16T00:38:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc9080a3240 0x7fc9080a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc91c106a80 0x7fc91c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc91c101420 0x7fc91c1989e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 --2- 172.21.15.31:0/2491715074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91c100a80 0x7fc91c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 >> 172.21.15.31:0/2491715074 conn(0x7fc91c0fc7a0 msgr2=0x7fc91c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 shutdown_connections 2024-09-16T00:38:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:56.010+0000 7fc923aa2640 1 -- 172.21.15.31:0/2491715074 wait complete. 2024-09-16T00:38:56.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:56 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2491715074' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:38:56.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:56 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2491715074' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:38:56.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:56 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2491715074' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:38:56.718 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T00:38:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:38:57.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:57 smithi031 ceph-mon[30956]: pgmap v280: 97 pgs: 13 creating+peering, 21 unknown, 63 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-09-16T00:38:57.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:57 smithi031 ceph-mon[30956]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:38:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:57 smithi031 ceph-mon[30956]: Cluster is now healthy 2024-09-16T00:38:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:57 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2491715074' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:38:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:57 smithi031 ceph-mon[30956]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:38:57.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:57 smithi102 ceph-mon[35364]: pgmap v280: 97 pgs: 13 creating+peering, 21 unknown, 63 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-09-16T00:38:57.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:57 smithi102 ceph-mon[35364]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:38:57.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:57 smithi102 ceph-mon[35364]: Cluster is now healthy 2024-09-16T00:38:57.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:57 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2491715074' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:38:57.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:57 smithi102 ceph-mon[35364]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:38:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:57 smithi081 ceph-mon[35539]: pgmap v280: 97 pgs: 13 creating+peering, 21 unknown, 63 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-09-16T00:38:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:57 smithi081 ceph-mon[35539]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:38:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:57 smithi081 ceph-mon[35539]: Cluster is now healthy 2024-09-16T00:38:57.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:57 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2491715074' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:38:57.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:57 smithi081 ceph-mon[35539]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:38:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2482177210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 msgr2=0x7f2c10107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2482177210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 0x7f2c10107b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009930 tx=0x7f2c0402a420 comp rx=0 tx=0).stop 2024-09-16T00:38:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2482177210 shutdown_connections 2024-09-16T00:38:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2482177210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c101082a0 0x7f2c1010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2482177210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 0x7f2c10107b90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2482177210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c10102d30 0x7f2c10103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2482177210 >> 172.21.15.31:0/2482177210 conn(0x7f2c100fc7d0 msgr2=0x7f2c100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2482177210 shutdown_connections 2024-09-16T00:38:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2482177210 wait complete. 2024-09-16T00:38:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.501+0000 7f2c14c5c640 1 Processor -- start 2024-09-16T00:38:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c14c5c640 1 -- start start 2024-09-16T00:38:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c14c5c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10102d30 0x7f2c10198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c14c5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 0x7f2c10198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c14c5c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c101082a0 0x7f2c1019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c14c5c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2c101120c0 con 0x7f2c101036e0 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c14c5c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10111f40 con 0x7f2c101082a0 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c14c5c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10112240 con 0x7f2c10102d30 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 0x7f2c10198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 0x7f2c10198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59028/0 (socket says 172.21.15.31:59028) 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0dd9b640 1 -- 172.21.15.31:0/2138910910 learned_addr learned my addr 172.21.15.31:0/2138910910 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0dd9b640 1 -- 172.21.15.31:0/2138910910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10102d30 msgr2=0x7f2c10198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0e59c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10102d30 0x7f2c10198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0ed9d640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c101082a0 0x7f2c1019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0dd9b640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10102d30 0x7f2c10198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0dd9b640 1 -- 172.21.15.31:0/2138910910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c101082a0 msgr2=0x7f2c1019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0dd9b640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c101082a0 0x7f2c1019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0dd9b640 1 -- 172.21.15.31:0/2138910910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1019d5d0 con 0x7f2c101036e0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0e59c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10102d30 0x7f2c10198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0ed9d640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c101082a0 0x7f2c1019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.502+0000 7f2c0dd9b640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 0x7f2c10198b60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2c04005940 tx=0x7f2c040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.503+0000 7f2bff7fe640 1 -- 172.21.15.31:0/2138910910 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0403f030 con 0x7f2c101036e0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.503+0000 7f2bff7fe640 1 -- 172.21.15.31:0/2138910910 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c04003ce0 con 0x7f2c101036e0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.503+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1019d860 con 0x7f2c101036e0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.503+0000 7f2bff7fe640 1 -- 172.21.15.31:0/2138910910 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04035070 con 0x7f2c101036e0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.503+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c101a51a0 con 0x7f2c101036e0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.504+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1006a850 con 0x7f2c101036e0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.505+0000 7f2bff7fe640 1 -- 172.21.15.31:0/2138910910 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2c04042030 con 0x7f2c101036e0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.505+0000 7f2bff7fe640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2bf80a3240 0x7f2bf80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.506+0000 7f2c0e59c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2bf80a3240 0x7f2bf80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.506+0000 7f2bff7fe640 1 -- 172.21.15.31:0/2138910910 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8277+0+0 (secure 0 0 0) 0x7f2c040efea0 con 0x7f2c101036e0 2024-09-16T00:38:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.506+0000 7f2c0e59c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2bf80a3240 0x7f2bf80a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2c00005fd0 tx=0x7f2c000093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.510+0000 7f2bff7fe640 1 -- 172.21.15.31:0/2138910910 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c040bcb90 con 0x7f2c101036e0 2024-09-16T00:38:57.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:57.686+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2c1010a2b0 con 0x7f2c101036e0 2024-09-16T00:38:58.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:58 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2138910910' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:38:58.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:58 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2138910910' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:38:58.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:58 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2138910910' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:38:58.497 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:38:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b[65567]: 2024-09-16T00:38:58.236+0000 7f3cb0063640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T00:38:59.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.024+0000 7f2bff7fe640 1 -- 172.21.15.31:0/2138910910 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v84) ==== 228+0+0 (secure 0 0 0) 0x7f2c040bc490 con 0x7f2c101036e0 2024-09-16T00:38:59.025 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data_ec' created 2024-09-16T00:38:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2bf80a3240 msgr2=0x7f2bf80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2bf80a3240 0x7f2bf80a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2c00005fd0 tx=0x7f2c000093e0 comp rx=0 tx=0).stop 2024-09-16T00:38:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 msgr2=0x7f2c10198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 0x7f2c10198b60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2c04005940 tx=0x7f2c040042f0 comp rx=0 tx=0).stop 2024-09-16T00:38:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 shutdown_connections 2024-09-16T00:38:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c101082a0 0x7f2c1019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c101036e0 0x7f2c10198b60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2bf80a3240 0x7f2bf80a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 --2- 172.21.15.31:0/2138910910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10102d30 0x7f2c10198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 >> 172.21.15.31:0/2138910910 conn(0x7f2c100fc7d0 msgr2=0x7f2c10105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.027+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 shutdown_connections 2024-09-16T00:38:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:38:59.028+0000 7f2c14c5c640 1 -- 172.21.15.31:0/2138910910 wait complete. 2024-09-16T00:38:59.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:59 smithi102 ceph-mon[35364]: pgmap v282: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 3 op/s 2024-09-16T00:38:59.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:38:59 smithi102 ceph-mon[35364]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:38:59.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:59 smithi031 ceph-mon[30956]: pgmap v282: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 3 op/s 2024-09-16T00:38:59.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:38:59 smithi031 ceph-mon[30956]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:38:59.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:59 smithi081 ceph-mon[35539]: pgmap v282: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 3 op/s 2024-09-16T00:38:59.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:38:59 smithi081 ceph-mon[35539]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:38:59.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-16T00:38:59.983 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:00.302 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:00 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2138910910' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:39:00.302 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:00 smithi102 ceph-mon[35364]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:39:00.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:00 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2138910910' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:39:00.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:00 smithi031 ceph-mon[30956]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:39:00.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:00 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2138910910' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:39:00.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:00 smithi081 ceph-mon[35539]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:39:00.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.512+0000 7fdb6175f640 1 -- 172.21.15.31:0/2541934161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c106a80 msgr2=0x7fdb5c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.512+0000 7fdb6175f640 1 --2- 172.21.15.31:0/2541934161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c106a80 0x7fdb5c106e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009930 tx=0x7fdb4c02a420 comp rx=0 tx=0).stop 2024-09-16T00:39:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.512+0000 7fdb6175f640 1 -- 172.21.15.31:0/2541934161 shutdown_connections 2024-09-16T00:39:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.512+0000 7fdb6175f640 1 --2- 172.21.15.31:0/2541934161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb5c101420 0x7fdb5c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.512+0000 7fdb6175f640 1 --2- 172.21.15.31:0/2541934161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c100a80 0x7fdb5c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.512+0000 7fdb6175f640 1 --2- 172.21.15.31:0/2541934161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c106a80 0x7fdb5c106e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.512+0000 7fdb6175f640 1 -- 172.21.15.31:0/2541934161 >> 172.21.15.31:0/2541934161 conn(0x7fdb5c0fc7a0 msgr2=0x7fdb5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.512+0000 7fdb6175f640 1 -- 172.21.15.31:0/2541934161 shutdown_connections 2024-09-16T00:39:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.513+0000 7fdb6175f640 1 -- 172.21.15.31:0/2541934161 wait complete. 2024-09-16T00:39:00.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.513+0000 7fdb6175f640 1 Processor -- start 2024-09-16T00:39:00.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.513+0000 7fdb6175f640 1 -- start start 2024-09-16T00:39:00.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb6175f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c100a80 0x7fdb5c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:00.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb6175f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb5c101420 0x7fdb5c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:00.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb6175f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c106a80 0x7fdb5c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:00.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb6175f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c110f20 con 0x7fdb5c100a80 2024-09-16T00:39:00.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb6175f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c110da0 con 0x7fdb5c106a80 2024-09-16T00:39:00.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb6175f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1110a0 con 0x7fdb5c101420 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c100a80 0x7fdb5c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c100a80 0x7fdb5c1974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59032/0 (socket says 172.21.15.31:59032) 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5affd640 1 -- 172.21.15.31:0/402297857 learned_addr learned my addr 172.21.15.31:0/402297857 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5b7fe640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c106a80 0x7fdb5c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5affd640 1 -- 172.21.15.31:0/402297857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb5c101420 msgr2=0x7fdb5c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb53fff640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb5c101420 0x7fdb5c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5affd640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb5c101420 0x7fdb5c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5affd640 1 -- 172.21.15.31:0/402297857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c106a80 msgr2=0x7fdb5c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5affd640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c106a80 0x7fdb5c19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5affd640 1 -- 172.21.15.31:0/402297857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c19c4a0 con 0x7fdb5c100a80 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5b7fe640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c106a80 0x7fdb5c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb5affd640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c100a80 0x7fdb5c1974f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009a50 tx=0x7fdb4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.514+0000 7fdb58ff9640 1 -- 172.21.15.31:0/402297857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c03f030 con 0x7fdb5c100a80 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.515+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c19c730 con 0x7fdb5c100a80 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.515+0000 7fdb58ff9640 1 -- 172.21.15.31:0/402297857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4c003ce0 con 0x7fdb5c100a80 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.515+0000 7fdb58ff9640 1 -- 172.21.15.31:0/402297857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c035070 con 0x7fdb5c100a80 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.515+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c1a4070 con 0x7fdb5c100a80 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.515+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5c100f60 con 0x7fdb5c100a80 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.517+0000 7fdb58ff9640 1 -- 172.21.15.31:0/402297857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fdb4c042030 con 0x7fdb5c100a80 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.518+0000 7fdb58ff9640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb480a3240 0x7fdb480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.518+0000 7fdb53fff640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb480a3240 0x7fdb480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.518+0000 7fdb58ff9640 1 -- 172.21.15.31:0/402297857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7fdb4c0f05d0 con 0x7fdb5c100a80 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.519+0000 7fdb53fff640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb480a3240 0x7fdb480a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0fedf0 tx=0x7fdb44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.522+0000 7fdb58ff9640 1 -- 172.21.15.31:0/402297857 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb4c0bd0d0 con 0x7fdb5c100a80 2024-09-16T00:39:00.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:00.697+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fdb5c198850 con 0x7fdb5c100a80 2024-09-16T00:39:01.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.035+0000 7fdb58ff9640 1 -- 172.21.15.31:0/402297857 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) ==== 169+0+0 (secure 0 0 0) 0x7fdb4c0bc9d0 con 0x7fdb5c100a80 2024-09-16T00:39:01.037 INFO:teuthology.orchestra.run.smithi031.stderr:set pool 4 allow_ec_overwrites to true 2024-09-16T00:39:01.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb480a3240 msgr2=0x7fdb480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:01.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb480a3240 0x7fdb480a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0fedf0 tx=0x7fdb44008040 comp rx=0 tx=0).stop 2024-09-16T00:39:01.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c100a80 msgr2=0x7fdb5c1974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:01.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c100a80 0x7fdb5c1974f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009a50 tx=0x7fdb4c004270 comp rx=0 tx=0).stop 2024-09-16T00:39:01.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 shutdown_connections 2024-09-16T00:39:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c106a80 0x7fdb5c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb480a3240 0x7fdb480a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb5c101420 0x7fdb5c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 --2- 172.21.15.31:0/402297857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb5c100a80 0x7fdb5c1974f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 >> 172.21.15.31:0/402297857 conn(0x7fdb5c0fc7a0 msgr2=0x7fdb5c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.038+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 shutdown_connections 2024-09-16T00:39:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:01.039+0000 7fdb6175f640 1 -- 172.21.15.31:0/402297857 wait complete. 2024-09-16T00:39:01.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:01 smithi102 ceph-mon[35364]: pgmap v285: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-16T00:39:01.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:01 smithi102 ceph-mon[35364]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:39:01.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:01 smithi102 ceph-mon[35364]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:39:01.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:01 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/402297857' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:39:01.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:01 smithi031 ceph-mon[30956]: pgmap v285: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-16T00:39:01.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:01 smithi031 ceph-mon[30956]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:39:01.374 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:01 smithi031 ceph-mon[30956]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:39:01.374 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:01 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/402297857' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:39:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:01 smithi081 ceph-mon[35539]: pgmap v285: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-16T00:39:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:01 smithi081 ceph-mon[35539]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:39:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:01 smithi081 ceph-mon[35539]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:39:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:01 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/402297857' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:39:01.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-16T00:39:01.976 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:02.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:02 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/402297857' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:39:02.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:02 smithi102 ceph-mon[35364]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:39:02.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:02 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/402297857' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:39:02.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:02 smithi031 ceph-mon[30956]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:39:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:02 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/402297857' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:39:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:02 smithi081 ceph-mon[35539]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:39:02.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.492+0000 7fd208c14640 1 -- 172.21.15.31:0/1619241940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204102d40 msgr2=0x7fd204103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.492+0000 7fd208c14640 1 --2- 172.21.15.31:0/1619241940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204102d40 0x7fd204103120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009900 tx=0x7fd1f402a480 comp rx=0 tx=0).stop 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.492+0000 7fd208c14640 1 -- 172.21.15.31:0/1619241940 shutdown_connections 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.492+0000 7fd208c14640 1 --2- 172.21.15.31:0/1619241940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd20410c170 0x7fd20410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.492+0000 7fd208c14640 1 --2- 172.21.15.31:0/1619241940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd204103660 0x7fd20410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.492+0000 7fd208c14640 1 --2- 172.21.15.31:0/1619241940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204102d40 0x7fd204103120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.492+0000 7fd208c14640 1 -- 172.21.15.31:0/1619241940 >> 172.21.15.31:0/1619241940 conn(0x7fd2040fc7a0 msgr2=0x7fd2040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.492+0000 7fd208c14640 1 -- 172.21.15.31:0/1619241940 shutdown_connections 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.492+0000 7fd208c14640 1 -- 172.21.15.31:0/1619241940 wait complete. 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.493+0000 7fd208c14640 1 Processor -- start 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.493+0000 7fd208c14640 1 -- start start 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.493+0000 7fd208c14640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd204102d40 0x7fd2041009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.493+0000 7fd208c14640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204103660 0x7fd204100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd208c14640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd20410c170 0x7fd204101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd208c14640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd204111fb0 con 0x7fd204103660 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd208c14640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd204111e30 con 0x7fd204102d40 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd208c14640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd204112130 con 0x7fd20410c170 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd201d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204103660 0x7fd204100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd201d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204103660 0x7fd204100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59048/0 (socket says 172.21.15.31:59048) 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd201d9b640 1 -- 172.21.15.31:0/2736323713 learned_addr learned my addr 172.21.15.31:0/2736323713 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd20259c640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd204102d40 0x7fd2041009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd202d9d640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd20410c170 0x7fd204101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd201d9b640 1 -- 172.21.15.31:0/2736323713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd20410c170 msgr2=0x7fd204101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd201d9b640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd20410c170 0x7fd204101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd201d9b640 1 -- 172.21.15.31:0/2736323713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd204102d40 msgr2=0x7fd2041009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd201d9b640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd204102d40 0x7fd2041009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:02.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd201d9b640 1 -- 172.21.15.31:0/2736323713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2041a6950 con 0x7fd204103660 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd202d9d640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd20410c170 0x7fd204101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd201d9b640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204103660 0x7fd204100ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800d740 tx=0x7fd1f800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd1f37fe640 1 -- 172.21.15.31:0/2736323713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8017070 con 0x7fd204103660 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2041a6c40 con 0x7fd204103660 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd1f37fe640 1 -- 172.21.15.31:0/2736323713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f800de50 con 0x7fd204103660 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd1f37fe640 1 -- 172.21.15.31:0/2736323713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f800f620 con 0x7fd204103660 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.494+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041a7180 con 0x7fd204103660 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.496+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd204103120 con 0x7fd204103660 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.497+0000 7fd1f37fe640 1 -- 172.21.15.31:0/2736323713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd1f8009c30 con 0x7fd204103660 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.497+0000 7fd1f37fe640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1ec0a3240 0x7fd1ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.497+0000 7fd20259c640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1ec0a3240 0x7fd1ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.497+0000 7fd1f37fe640 1 -- 172.21.15.31:0/2736323713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7fd1f80cf7a0 con 0x7fd204103660 2024-09-16T00:39:02.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.498+0000 7fd20259c640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1ec0a3240 0x7fd1ec0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4005ab0 tx=0x7fd1f4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:02.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.502+0000 7fd1f37fe640 1 -- 172.21.15.31:0/2736323713 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1f809c2a0 con 0x7fd204103660 2024-09-16T00:39:02.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:02.714+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fd204106c80 con 0x7fd204103660 2024-09-16T00:39:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.047+0000 7fd1f37fe640 1 -- 172.21.15.31:0/2736323713 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7fd1f809bba0 con 0x7fd204103660 2024-09-16T00:39:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:added data pool 4 to fsmap 2024-09-16T00:39:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1ec0a3240 msgr2=0x7fd1ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1ec0a3240 0x7fd1ec0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4005ab0 tx=0x7fd1f4005a20 comp rx=0 tx=0).stop 2024-09-16T00:39:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204103660 msgr2=0x7fd204100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204103660 0x7fd204100ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800d740 tx=0x7fd1f800dc00 comp rx=0 tx=0).stop 2024-09-16T00:39:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 shutdown_connections 2024-09-16T00:39:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1ec0a3240 0x7fd1ec0a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd20410c170 0x7fd204101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd204103660 0x7fd204100ee0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 --2- 172.21.15.31:0/2736323713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd204102d40 0x7fd2041009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 >> 172.21.15.31:0/2736323713 conn(0x7fd2040fc7a0 msgr2=0x7fd2040fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 shutdown_connections 2024-09-16T00:39:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:03.050+0000 7fd208c14640 1 -- 172.21.15.31:0/2736323713 wait complete. 2024-09-16T00:39:03.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:03 smithi102 ceph-mon[35364]: pgmap v288: 129 pgs: 4 creating+peering, 16 unknown, 109 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-09-16T00:39:03.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:03 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2736323713' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:39:03.365 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:03 smithi031 ceph-mon[30956]: pgmap v288: 129 pgs: 4 creating+peering, 16 unknown, 109 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-09-16T00:39:03.365 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:03 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2736323713' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:39:03.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:03 smithi081 ceph-mon[35539]: pgmap v288: 129 pgs: 4 creating+peering, 16 unknown, 109 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-09-16T00:39:03.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:03 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2736323713' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:39:03.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:39:03.974 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:04.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:04 smithi102 ceph-mon[35364]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:39:04.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:04 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2736323713' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:39:04.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:04 smithi102 ceph-mon[35364]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:39:04.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:04 smithi031 ceph-mon[30956]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:39:04.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:04 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2736323713' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:39:04.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:04 smithi031 ceph-mon[30956]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:39:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.436+0000 7f2434719640 1 -- 172.21.15.31:0/3370140625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c106f10 msgr2=0x7f242c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.436+0000 7f2434719640 1 --2- 172.21.15.31:0/3370140625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c106f10 0x7f242c10ea90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2424009ca0 tx=0x7f242402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.436+0000 7f2434719640 1 -- 172.21.15.31:0/3370140625 shutdown_connections 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.436+0000 7f2434719640 1 --2- 172.21.15.31:0/3370140625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c106f10 0x7f242c10ea90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.436+0000 7f2434719640 1 --2- 172.21.15.31:0/3370140625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f242c102790 0x7f242c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.436+0000 7f2434719640 1 --2- 172.21.15.31:0/3370140625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f242c101de0 0x7f242c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.436+0000 7f2434719640 1 -- 172.21.15.31:0/3370140625 >> 172.21.15.31:0/3370140625 conn(0x7f242c0fd8c0 msgr2=0x7f242c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.436+0000 7f2434719640 1 -- 172.21.15.31:0/3370140625 shutdown_connections 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.436+0000 7f2434719640 1 -- 172.21.15.31:0/3370140625 wait complete. 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2434719640 1 Processor -- start 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2434719640 1 -- start start 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2434719640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c101de0 0x7f242c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2434719640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f242c102790 0x7f242c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2434719640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f242c106f10 0x7f242c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2434719640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f242c111f70 con 0x7f242c101de0 2024-09-16T00:39:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2434719640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f242c111df0 con 0x7f242c106f10 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2434719640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f242c1120f0 con 0x7f242c102790 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f24324b5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c101de0 0x7f242c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f24324b5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c101de0 0x7f242c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59054/0 (socket says 172.21.15.31:59054) 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f24324b5640 1 -- 172.21.15.31:0/1564295355 learned_addr learned my addr 172.21.15.31:0/1564295355 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2431cb4640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f242c102790 0x7f242c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f24324b5640 1 -- 172.21.15.31:0/1564295355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f242c102790 msgr2=0x7f242c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f24324b5640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f242c102790 0x7f242c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f2432cb6640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f242c106f10 0x7f242c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f24324b5640 1 -- 172.21.15.31:0/1564295355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f242c106f10 msgr2=0x7f242c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f24324b5640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f242c106f10 0x7f242c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.437+0000 7f24324b5640 1 -- 172.21.15.31:0/1564295355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c19f590 con 0x7f242c101de0 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.438+0000 7f24324b5640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c101de0 0x7f242c19a5e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f242800d770 tx=0x7f242800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.438+0000 7f24237fe640 1 -- 172.21.15.31:0/1564295355 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428017070 con 0x7f242c101de0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.438+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c19f880 con 0x7f242c101de0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.438+0000 7f24237fe640 1 -- 172.21.15.31:0/1564295355 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2428005500 con 0x7f242c101de0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.438+0000 7f24237fe640 1 -- 172.21.15.31:0/1564295355 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428004e00 con 0x7f242c101de0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.438+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c1a7160 con 0x7f242c101de0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.439+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c1021c0 con 0x7f242c101de0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.440+0000 7f24237fe640 1 -- 172.21.15.31:0/1564295355 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2428004fa0 con 0x7f242c101de0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.441+0000 7f24237fe640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f24180a3240 0x7f24180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.441+0000 7f2431cb4640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f24180a3240 0x7f24180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.441+0000 7f24237fe640 1 -- 172.21.15.31:0/1564295355 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f24280cfa00 con 0x7f242c101de0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.441+0000 7f2431cb4640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f24180a3240 0x7f24180a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f242c0ff690 tx=0x7f241c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.444+0000 7f24237fe640 1 -- 172.21.15.31:0/1564295355 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f242809c4e0 con 0x7f242c101de0 2024-09-16T00:39:04.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:04 smithi081 ceph-mon[35539]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:39:04.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:04 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2736323713' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:39:04.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:04 smithi081 ceph-mon[35539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:39:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.654+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f242c1a7390 con 0x7f242c101de0 2024-09-16T00:39:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.655+0000 7f24237fe640 1 -- 172.21.15.31:0/1564295355 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11839 (secure 0 0 0) 0x7f242809bde0 con 0x7f242c101de0 2024-09-16T00:39:04.655 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:04.656 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"btime":"2024-09-16T00:39:03:042959+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:02.716650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:39:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 16 2024-09-16T00:39:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f24180a3240 msgr2=0x7f24180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f24180a3240 0x7f24180a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f242c0ff690 tx=0x7f241c00b390 comp rx=0 tx=0).stop 2024-09-16T00:39:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c101de0 msgr2=0x7f242c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c101de0 0x7f242c19a5e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f242800d770 tx=0x7f242800dc30 comp rx=0 tx=0).stop 2024-09-16T00:39:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 shutdown_connections 2024-09-16T00:39:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f242c106f10 0x7f242c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f24180a3240 0x7f24180a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f242c102790 0x7f242c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 --2- 172.21.15.31:0/1564295355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f242c101de0 0x7f242c19a5e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 >> 172.21.15.31:0/1564295355 conn(0x7f242c0fd8c0 msgr2=0x7f242c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 shutdown_connections 2024-09-16T00:39:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:04.657+0000 7f2434719640 1 -- 172.21.15.31:0/1564295355 wait complete. 2024-09-16T00:39:05.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:05 smithi102 ceph-mon[35364]: pgmap v290: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T00:39:05.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:05 smithi102 ceph-mon[35364]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:39:05.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:05 smithi102 ceph-mon[35364]: Cluster is now healthy 2024-09-16T00:39:05.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:05 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1564295355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:05.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:05 smithi031 ceph-mon[30956]: pgmap v290: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T00:39:05.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:05 smithi031 ceph-mon[30956]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:39:05.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:05 smithi031 ceph-mon[30956]: Cluster is now healthy 2024-09-16T00:39:05.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:05 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1564295355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:05.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:39:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:05 smithi081 ceph-mon[35539]: pgmap v290: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T00:39:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:05 smithi081 ceph-mon[35539]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:39:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:05 smithi081 ceph-mon[35539]: Cluster is now healthy 2024-09-16T00:39:05.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:05 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1564295355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:05.620 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.144+0000 7f9dd827a640 1 -- 172.21.15.31:0/2631890391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 msgr2=0x7f9dd010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.144+0000 7f9dd827a640 1 --2- 172.21.15.31:0/2631890391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 0x7f9dd010e600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009ca0 tx=0x7f9dc802a950 comp rx=0 tx=0).stop 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.144+0000 7f9dd827a640 1 -- 172.21.15.31:0/2631890391 shutdown_connections 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.144+0000 7f9dd827a640 1 --2- 172.21.15.31:0/2631890391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 0x7f9dd010e600 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.144+0000 7f9dd827a640 1 --2- 172.21.15.31:0/2631890391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dd01036c0 0x7f9dd010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.144+0000 7f9dd827a640 1 --2- 172.21.15.31:0/2631890391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd0102da0 0x7f9dd0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.144+0000 7f9dd827a640 1 -- 172.21.15.31:0/2631890391 >> 172.21.15.31:0/2631890391 conn(0x7f9dd00fc7a0 msgr2=0x7f9dd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.144+0000 7f9dd827a640 1 -- 172.21.15.31:0/2631890391 shutdown_connections 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.144+0000 7f9dd827a640 1 -- 172.21.15.31:0/2631890391 wait complete. 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd827a640 1 Processor -- start 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd827a640 1 -- start start 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd827a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd0102da0 0x7f9dd0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd827a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dd01036c0 0x7f9dd0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd827a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 0x7f9dd019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd827a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0111fa0 con 0x7f9dd010c180 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd827a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0111e20 con 0x7f9dd0102da0 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd827a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0112120 con 0x7f9dd01036c0 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd6817640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 0x7f9dd019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd6817640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 0x7f9dd019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59084/0 (socket says 172.21.15.31:59084) 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd6817640 1 -- 172.21.15.31:0/1158539834 learned_addr learned my addr 172.21.15.31:0/1158539834 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.145+0000 7f9dd6016640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd0102da0 0x7f9dd0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dd5815640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dd01036c0 0x7f9dd0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dd6817640 1 -- 172.21.15.31:0/1158539834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dd01036c0 msgr2=0x7f9dd0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dd6817640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dd01036c0 0x7f9dd0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dd6817640 1 -- 172.21.15.31:0/1158539834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd0102da0 msgr2=0x7f9dd0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dd6817640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd0102da0 0x7f9dd0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dd6817640 1 -- 172.21.15.31:0/1158539834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd019d3f0 con 0x7f9dd010c180 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dd6817640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 0x7f9dd019cd10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9dc802ae70 tx=0x7f9dc802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dc77fe640 1 -- 172.21.15.31:0/1158539834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8034040 con 0x7f9dd010c180 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd019d680 con 0x7f9dd010c180 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dc77fe640 1 -- 172.21.15.31:0/1158539834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc8030920 con 0x7f9dd010c180 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dc77fe640 1 -- 172.21.15.31:0/1158539834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8045480 con 0x7f9dd010c180 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.146+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd01a4fc0 con 0x7f9dd010c180 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.147+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd006b010 con 0x7f9dd010c180 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.149+0000 7f9dc77fe640 1 -- 172.21.15.31:0/1158539834 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9dc8037020 con 0x7f9dd010c180 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.150+0000 7f9dc77fe640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9dbc0a3240 0x7f9dbc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.150+0000 7f9dc77fe640 1 -- 172.21.15.31:0/1158539834 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f9dc80f0c10 con 0x7f9dd010c180 2024-09-16T00:39:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.150+0000 7f9dd6016640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9dbc0a3240 0x7f9dbc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:06.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.150+0000 7f9dd6016640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9dbc0a3240 0x7f9dbc0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00bbe0 tx=0x7f9dcc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:06.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.153+0000 7f9dc77fe640 1 -- 172.21.15.31:0/1158539834 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9dc80811b0 con 0x7f9dd010c180 2024-09-16T00:39:06.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.330+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dd0107200 con 0x7f9dd010c180 2024-09-16T00:39:06.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.332+0000 7f9dc77fe640 1 -- 172.21.15.31:0/1158539834 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24150 (secure 0 0 0) 0x7f9dc80bd6f0 con 0x7f9dd010c180 2024-09-16T00:39:06.333 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:06.333 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:39:02.716519+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:39:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9dbc0a3240 msgr2=0x7f9dbc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9dbc0a3240 0x7f9dbc0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00bbe0 tx=0x7f9dcc00b390 comp rx=0 tx=0).stop 2024-09-16T00:39:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 msgr2=0x7f9dd019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:06.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 0x7f9dd019cd10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9dc802ae70 tx=0x7f9dc802f5b0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 shutdown_connections 2024-09-16T00:39:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd010c180 0x7f9dd019cd10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9dbc0a3240 0x7f9dbc0a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dd01036c0 0x7f9dd0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 --2- 172.21.15.31:0/1158539834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd0102da0 0x7f9dd0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 >> 172.21.15.31:0/1158539834 conn(0x7f9dd00fc7a0 msgr2=0x7f9dd00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 shutdown_connections 2024-09-16T00:39:06.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:06.335+0000 7f9dd827a640 1 -- 172.21.15.31:0/1158539834 wait complete. 2024-09-16T00:39:07.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:39:07.300 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:07 smithi081 ceph-mon[35539]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:07.300 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:07 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1158539834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:07.317 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:07.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:07 smithi031 ceph-mon[30956]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:07.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:07 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1158539834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:07.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:07 smithi102 ceph-mon[35364]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:07.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:07 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1158539834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:07.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.777+0000 7f20de9d8640 1 -- 172.21.15.31:0/890559507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d8103660 msgr2=0x7f20d810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:07.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.777+0000 7f20de9d8640 1 --2- 172.21.15.31:0/890559507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d8103660 0x7f20d810bb50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009900 tx=0x7f20cc02a480 comp rx=0 tx=0).stop 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.777+0000 7f20de9d8640 1 -- 172.21.15.31:0/890559507 shutdown_connections 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.777+0000 7f20de9d8640 1 --2- 172.21.15.31:0/890559507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20d810c170 0x7f20d810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.777+0000 7f20de9d8640 1 --2- 172.21.15.31:0/890559507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d8103660 0x7f20d810bb50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.777+0000 7f20de9d8640 1 --2- 172.21.15.31:0/890559507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20d8102d40 0x7f20d8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.777+0000 7f20de9d8640 1 -- 172.21.15.31:0/890559507 >> 172.21.15.31:0/890559507 conn(0x7f20d80fc7a0 msgr2=0x7f20d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.777+0000 7f20de9d8640 1 -- 172.21.15.31:0/890559507 shutdown_connections 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.777+0000 7f20de9d8640 1 -- 172.21.15.31:0/890559507 wait complete. 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20de9d8640 1 Processor -- start 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20de9d8640 1 -- start start 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20de9d8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20d8102d40 0x7f20d81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20de9d8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20d8103660 0x7f20d8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20de9d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d810c170 0x7f20d8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20de9d8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8111f70 con 0x7f20d810c170 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20de9d8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8111df0 con 0x7f20d8102d40 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20de9d8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81120f0 con 0x7f20d8103660 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20dcf75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d810c170 0x7f20d8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20dcf75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d810c170 0x7f20d8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35576/0 (socket says 172.21.15.31:35576) 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20dcf75640 1 -- 172.21.15.31:0/3526405315 learned_addr learned my addr 172.21.15.31:0/3526405315 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20d7fff640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20d8102d40 0x7f20d81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20d77fe640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20d8103660 0x7f20d8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20dcf75640 1 -- 172.21.15.31:0/3526405315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20d8103660 msgr2=0x7f20d8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20dcf75640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20d8103660 0x7f20d8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20dcf75640 1 -- 172.21.15.31:0/3526405315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20d8102d40 msgr2=0x7f20d81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20dcf75640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20d8102d40 0x7f20d81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20dcf75640 1 -- 172.21.15.31:0/3526405315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d81a6910 con 0x7f20d810c170 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.778+0000 7f20d77fe640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20d8103660 0x7f20d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.779+0000 7f20dcf75640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d810c170 0x7f20d8101420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20d000eaa0 tx=0x7f20d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.779+0000 7f20d57fa640 1 -- 172.21.15.31:0/3526405315 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d000cdb0 con 0x7f20d810c170 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.779+0000 7f20d57fa640 1 -- 172.21.15.31:0/3526405315 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d0005020 con 0x7f20d810c170 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.779+0000 7f20d57fa640 1 -- 172.21.15.31:0/3526405315 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00116b0 con 0x7f20d810c170 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.779+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d81a6c00 con 0x7f20d810c170 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.779+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d81a70f0 con 0x7f20d810c170 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.780+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d8103120 con 0x7f20d810c170 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.781+0000 7f20d57fa640 1 -- 172.21.15.31:0/3526405315 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f20d0005560 con 0x7f20d810c170 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.782+0000 7f20d57fa640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20c00a3240 0x7f20c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.782+0000 7f20d57fa640 1 -- 172.21.15.31:0/3526405315 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f20d00cfd60 con 0x7f20d810c170 2024-09-16T00:39:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.782+0000 7f20d7fff640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20c00a3240 0x7f20c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:07.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.782+0000 7f20d7fff640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20c00a3240 0x7f20c00a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005fd0 tx=0x7f20c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:07.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.786+0000 7f20d57fa640 1 -- 172.21.15.31:0/3526405315 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20d0016030 con 0x7f20d810c170 2024-09-16T00:39:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.962+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20d8106c50 con 0x7f20d810c170 2024-09-16T00:39:07.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.964+0000 7f20d57fa640 1 -- 172.21.15.31:0/3526405315 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24150 (secure 0 0 0) 0x7f20d009c840 con 0x7f20d810c170 2024-09-16T00:39:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:07.964 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:39:02.716519+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:39:07.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20c00a3240 msgr2=0x7f20c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:07.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20c00a3240 0x7f20c00a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005fd0 tx=0x7f20c800b390 comp rx=0 tx=0).stop 2024-09-16T00:39:07.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d810c170 msgr2=0x7f20d8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:07.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d810c170 0x7f20d8101420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20d000eaa0 tx=0x7f20d000ef60 comp rx=0 tx=0).stop 2024-09-16T00:39:07.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 shutdown_connections 2024-09-16T00:39:07.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20d810c170 0x7f20d8101420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:07.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20c00a3240 0x7f20c00a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:07.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20d8103660 0x7f20d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:07.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 --2- 172.21.15.31:0/3526405315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20d8102d40 0x7f20d81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:07.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 >> 172.21.15.31:0/3526405315 conn(0x7f20d80fc7a0 msgr2=0x7f20d80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:07.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 shutdown_connections 2024-09-16T00:39:07.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:07.966+0000 7f20de9d8640 1 -- 172.21.15.31:0/3526405315 wait complete. 2024-09-16T00:39:08.283 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:08 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3526405315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:08.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:08 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3526405315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:08.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:08 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3526405315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:08.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:39:08.994 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:09.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:09 smithi031 ceph-mon[30956]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:09.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:09 smithi081 ceph-mon[35539]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.484+0000 7fc6c8d01640 1 -- 172.21.15.31:0/1328238634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 msgr2=0x7fc6c410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.484+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/1328238634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 0x7fc6c410da40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009ca0 tx=0x7fc6bc02a950 comp rx=0 tx=0).stop 2024-09-16T00:39:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.485+0000 7fc6c8d01640 1 -- 172.21.15.31:0/1328238634 shutdown_connections 2024-09-16T00:39:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.485+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/1328238634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 0x7fc6c410da40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.485+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/1328238634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6c4102570 0x7fc6c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.485+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/1328238634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6c4100e20 0x7fc6c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.485+0000 7fc6c8d01640 1 -- 172.21.15.31:0/1328238634 >> 172.21.15.31:0/1328238634 conn(0x7fc6c40fc7d0 msgr2=0x7fc6c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.485+0000 7fc6c8d01640 1 -- 172.21.15.31:0/1328238634 shutdown_connections 2024-09-16T00:39:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.485+0000 7fc6c8d01640 1 -- 172.21.15.31:0/1328238634 wait complete. 2024-09-16T00:39:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.485+0000 7fc6c8d01640 1 Processor -- start 2024-09-16T00:39:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.485+0000 7fc6c8d01640 1 -- start start 2024-09-16T00:39:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c8d01640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6c4100e20 0x7fc6c4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c8d01640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6c4102570 0x7fc6c4198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c8d01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 0x7fc6c419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c8d01640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41120a0 con 0x7fc6c4102ef0 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c8d01640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4111f20 con 0x7fc6c4102570 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c8d01640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4112220 con 0x7fc6c4100e20 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 0x7fc6c419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6c4100e20 0x7fc6c4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 0x7fc6c419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35596/0 (socket says 172.21.15.31:35596) 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c2d9d640 1 -- 172.21.15.31:0/2280609662 learned_addr learned my addr 172.21.15.31:0/2280609662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c1d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6c4102570 0x7fc6c4198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:09.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c2d9d640 1 -- 172.21.15.31:0/2280609662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6c4100e20 msgr2=0x7fc6c4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c2d9d640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6c4100e20 0x7fc6c4198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c2d9d640 1 -- 172.21.15.31:0/2280609662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6c4102570 msgr2=0x7fc6c4198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c2d9d640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6c4102570 0x7fc6c4198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c2d9d640 1 -- 172.21.15.31:0/2280609662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c419d4f0 con 0x7fc6c4102ef0 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6c2d9d640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 0x7fc6c419ce10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc02ae70 tx=0x7fc6bc02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.486+0000 7fc6b37fe640 1 -- 172.21.15.31:0/2280609662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc034040 con 0x7fc6c4102ef0 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.487+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c419d780 con 0x7fc6c4102ef0 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.487+0000 7fc6b37fe640 1 -- 172.21.15.31:0/2280609662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6bc030920 con 0x7fc6c4102ef0 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.487+0000 7fc6b37fe640 1 -- 172.21.15.31:0/2280609662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc0047f0 con 0x7fc6c4102ef0 2024-09-16T00:39:09.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.487+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c41a50c0 con 0x7fc6c4102ef0 2024-09-16T00:39:09.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.488+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c4101200 con 0x7fc6c4102ef0 2024-09-16T00:39:09.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.489+0000 7fc6b37fe640 1 -- 172.21.15.31:0/2280609662 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc6bc037020 con 0x7fc6c4102ef0 2024-09-16T00:39:09.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.489+0000 7fc6b37fe640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6ac0a32c0 0x7fc6ac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:09.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.489+0000 7fc6c259c640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6ac0a32c0 0x7fc6ac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:09.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.489+0000 7fc6b37fe640 1 -- 172.21.15.31:0/2280609662 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fc6bc0f0960 con 0x7fc6c4102ef0 2024-09-16T00:39:09.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.490+0000 7fc6c259c640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6ac0a32c0 0x7fc6ac0a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009990 tx=0x7fc6b4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:09.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.493+0000 7fc6b37fe640 1 -- 172.21.15.31:0/2280609662 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6bc0bd440 con 0x7fc6c4102ef0 2024-09-16T00:39:09.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:09 smithi102 ceph-mon[35364]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:09.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.705+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6c41a5290 con 0x7fc6c4102ef0 2024-09-16T00:39:09.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.706+0000 7fc6b37fe640 1 -- 172.21.15.31:0/2280609662 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11839 (secure 0 0 0) 0x7fc6bc0bcd40 con 0x7fc6c4102ef0 2024-09-16T00:39:09.706 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:09.706 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"btime":"2024-09-16T00:39:03:042959+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:02.716650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:39:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 16 2024-09-16T00:39:09.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6ac0a32c0 msgr2=0x7fc6ac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:09.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6ac0a32c0 0x7fc6ac0a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009990 tx=0x7fc6b4009110 comp rx=0 tx=0).stop 2024-09-16T00:39:09.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 msgr2=0x7fc6c419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:09.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 0x7fc6c419ce10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc02ae70 tx=0x7fc6bc02f5d0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 shutdown_connections 2024-09-16T00:39:09.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c4102ef0 0x7fc6c419ce10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6c4102570 0x7fc6c4198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6ac0a32c0 0x7fc6ac0a5780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 --2- 172.21.15.31:0/2280609662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6c4100e20 0x7fc6c4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 >> 172.21.15.31:0/2280609662 conn(0x7fc6c40fc7d0 msgr2=0x7fc6c4106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 shutdown_connections 2024-09-16T00:39:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:09.708+0000 7fc6c8d01640 1 -- 172.21.15.31:0/2280609662 wait complete. 2024-09-16T00:39:10.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:10 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2280609662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:10.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:10 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2280609662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:10.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:39:10.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:10 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2280609662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:11.173 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:11 smithi031 ceph-mon[30956]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:11.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.255+0000 7f3f956d5640 1 -- 172.21.15.31:0/790793768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90106f10 msgr2=0x7f3f9010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.255+0000 7f3f956d5640 1 --2- 172.21.15.31:0/790793768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90106f10 0x7f3f9010ea90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009ca0 tx=0x7f3f8802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.255+0000 7f3f956d5640 1 -- 172.21.15.31:0/790793768 shutdown_connections 2024-09-16T00:39:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.255+0000 7f3f956d5640 1 --2- 172.21.15.31:0/790793768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90106f10 0x7f3f9010ea90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.255+0000 7f3f956d5640 1 --2- 172.21.15.31:0/790793768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90102790 0x7f3f901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.255+0000 7f3f956d5640 1 --2- 172.21.15.31:0/790793768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f90101de0 0x7f3f901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.255+0000 7f3f956d5640 1 -- 172.21.15.31:0/790793768 >> 172.21.15.31:0/790793768 conn(0x7f3f900fd8c0 msgr2=0x7f3f900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.255+0000 7f3f956d5640 1 -- 172.21.15.31:0/790793768 shutdown_connections 2024-09-16T00:39:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.255+0000 7f3f956d5640 1 -- 172.21.15.31:0/790793768 wait complete. 2024-09-16T00:39:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.256+0000 7f3f956d5640 1 Processor -- start 2024-09-16T00:39:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.256+0000 7f3f956d5640 1 -- start start 2024-09-16T00:39:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.256+0000 7f3f956d5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90101de0 0x7f3f90198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.256+0000 7f3f956d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90102790 0x7f3f901989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.256+0000 7f3f956d5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f90106f10 0x7f3f9019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.256+0000 7f3f956d5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90111f90 con 0x7f3f90102790 2024-09-16T00:39:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.256+0000 7f3f956d5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90111e10 con 0x7f3f90106f10 2024-09-16T00:39:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.256+0000 7f3f956d5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90112110 con 0x7f3f90101de0 2024-09-16T00:39:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90102790 0x7f3f901989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90102790 0x7f3f901989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35608/0 (socket says 172.21.15.31:35608) 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8e7fc640 1 -- 172.21.15.31:0/741342997 learned_addr learned my addr 172.21.15.31:0/741342997 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8effd640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90101de0 0x7f3f90198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8f7fe640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f90106f10 0x7f3f9019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8e7fc640 1 -- 172.21.15.31:0/741342997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90101de0 msgr2=0x7f3f90198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8e7fc640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90101de0 0x7f3f90198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8e7fc640 1 -- 172.21.15.31:0/741342997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f90106f10 msgr2=0x7f3f9019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8e7fc640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f90106f10 0x7f3f9019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8e7fc640 1 -- 172.21.15.31:0/741342997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9019d420 con 0x7f3f90102790 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8effd640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90101de0 0x7f3f90198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f8e7fc640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90102790 0x7f3f901989b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400b4a0 tx=0x7f3f8400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f77fff640 1 -- 172.21.15.31:0/741342997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84014070 con 0x7f3f90102790 2024-09-16T00:39:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9019d710 con 0x7f3f90102790 2024-09-16T00:39:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f77fff640 1 -- 172.21.15.31:0/741342997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f84010af0 con 0x7f3f90102790 2024-09-16T00:39:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f77fff640 1 -- 172.21.15.31:0/741342997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84017d10 con 0x7f3f90102790 2024-09-16T00:39:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.257+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f901a4ff0 con 0x7f3f90102790 2024-09-16T00:39:11.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.258+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f901021c0 con 0x7f3f90102790 2024-09-16T00:39:11.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.259+0000 7f3f77fff640 1 -- 172.21.15.31:0/741342997 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3f84017070 con 0x7f3f90102790 2024-09-16T00:39:11.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.260+0000 7f3f77fff640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3f780a3240 0x7f3f780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:11.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.260+0000 7f3f8effd640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3f780a3240 0x7f3f780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:11.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.260+0000 7f3f77fff640 1 -- 172.21.15.31:0/741342997 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f3f840cf040 con 0x7f3f90102790 2024-09-16T00:39:11.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.261+0000 7f3f8effd640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3f780a3240 0x7f3f780a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3f800058c0 tx=0x7f3f80005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.265+0000 7f3f77fff640 1 -- 172.21.15.31:0/741342997 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f8409bb20 con 0x7f3f90102790 2024-09-16T00:39:11.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.440+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f90109b60 con 0x7f3f90102790 2024-09-16T00:39:11.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.442+0000 7f3f77fff640 1 -- 172.21.15.31:0/741342997 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24150 (secure 0 0 0) 0x7f3f8409b420 con 0x7f3f90102790 2024-09-16T00:39:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:11.442 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:39:02.716519+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:39:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3f780a3240 msgr2=0x7f3f780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3f780a3240 0x7f3f780a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3f800058c0 tx=0x7f3f80005ec0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90102790 msgr2=0x7f3f901989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90102790 0x7f3f901989b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400b4a0 tx=0x7f3f8400b960 comp rx=0 tx=0).stop 2024-09-16T00:39:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 shutdown_connections 2024-09-16T00:39:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f90106f10 0x7f3f9019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f90102790 0x7f3f901989b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3f780a3240 0x7f3f780a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 --2- 172.21.15.31:0/741342997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90101de0 0x7f3f90198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 >> 172.21.15.31:0/741342997 conn(0x7f3f900fd8c0 msgr2=0x7f3f900fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 shutdown_connections 2024-09-16T00:39:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:11.444+0000 7f3f956d5640 1 -- 172.21.15.31:0/741342997 wait complete. 2024-09-16T00:39:11.450 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:11 smithi102 ceph-mon[35364]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:11.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:11 smithi081 ceph-mon[35539]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:12.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T00:38:53.576781+0000', 'modified': '2024-09-16T00:39:02.716650+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6839/2243721734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 2243721734}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 2243721734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24482, 'qdb_cluster': [24482]} 2024-09-16T00:39:12.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T00:39:12.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:39:12.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:12 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/741342997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:12.462 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:12 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/741342997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:12.493 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:12.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:12 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/741342997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:12.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.957+0000 7f31061a1640 1 -- 172.21.15.31:0/1197203641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100102790 msgr2=0x7f31001067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.957+0000 7f31061a1640 1 --2- 172.21.15.31:0/1197203641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100102790 0x7f31001067e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009900 tx=0x7f30f402a410 comp rx=0 tx=0).stop 2024-09-16T00:39:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.957+0000 7f31061a1640 1 -- 172.21.15.31:0/1197203641 shutdown_connections 2024-09-16T00:39:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.957+0000 7f31061a1640 1 --2- 172.21.15.31:0/1197203641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3100106f10 0x7f310010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.957+0000 7f31061a1640 1 --2- 172.21.15.31:0/1197203641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100102790 0x7f31001067e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.957+0000 7f31061a1640 1 --2- 172.21.15.31:0/1197203641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100101de0 0x7f31001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.957+0000 7f31061a1640 1 -- 172.21.15.31:0/1197203641 >> 172.21.15.31:0/1197203641 conn(0x7f31000fd8c0 msgr2=0x7f31000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.957+0000 7f31061a1640 1 -- 172.21.15.31:0/1197203641 shutdown_connections 2024-09-16T00:39:12.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.957+0000 7f31061a1640 1 -- 172.21.15.31:0/1197203641 wait complete. 2024-09-16T00:39:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f31061a1640 1 Processor -- start 2024-09-16T00:39:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f31061a1640 1 -- start start 2024-09-16T00:39:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f31061a1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100101de0 0x7f3100198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f31061a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3100102790 0x7f31001989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f31061a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100106f10 0x7f310019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f31061a1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3100111f90 con 0x7f3100106f10 2024-09-16T00:39:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f31061a1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3100111e10 con 0x7f3100102790 2024-09-16T00:39:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f31061a1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3100112110 con 0x7f3100101de0 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f30fffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100106f10 0x7f310019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f30fffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100106f10 0x7f310019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35632/0 (socket says 172.21.15.31:35632) 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f30ff7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100101de0 0x7f3100198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.958+0000 7f30fffff640 1 -- 172.21.15.31:0/1292000552 learned_addr learned my addr 172.21.15.31:0/1292000552 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30feffd640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3100102790 0x7f31001989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30fffff640 1 -- 172.21.15.31:0/1292000552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100101de0 msgr2=0x7f3100198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30fffff640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100101de0 0x7f3100198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30fffff640 1 -- 172.21.15.31:0/1292000552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3100102790 msgr2=0x7f31001989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30fffff640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3100102790 0x7f31001989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30fffff640 1 -- 172.21.15.31:0/1292000552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310019d440 con 0x7f3100106f10 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30fffff640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100106f10 0x7f310019cd60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f30f800b970 tx=0x7f30f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30fcff9640 1 -- 172.21.15.31:0/1292000552 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8019070 con 0x7f3100106f10 2024-09-16T00:39:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30fcff9640 1 -- 172.21.15.31:0/1292000552 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f800cb40 con 0x7f3100106f10 2024-09-16T00:39:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f31061a1640 1 -- 172.21.15.31:0/1292000552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310019d730 con 0x7f3100106f10 2024-09-16T00:39:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f30fcff9640 1 -- 172.21.15.31:0/1292000552 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8007650 con 0x7f3100106f10 2024-09-16T00:39:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.959+0000 7f31061a1640 1 -- 172.21.15.31:0/1292000552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31001a5010 con 0x7f3100106f10 2024-09-16T00:39:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f310857a740 1 Processor -- start 2024-09-16T00:39:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f30fcff9640 1 -- 172.21.15.31:0/1292000552 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f30f801c070 con 0x7f3100106f10 2024-09-16T00:39:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f310857a740 1 -- start start 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f310857a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969219d0 0x563296921db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f310857a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x56329692c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f310857a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x56329692efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f310857a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563296888090 con 0x5632969219d0 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f310857a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56329682fa70 con 0x56329692cbb0 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f310857a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563296884050 con 0x5632969222f0 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f30feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969219d0 0x563296921db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f30feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969219d0 0x563296921db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35640/0 (socket says 172.21.15.31:35640) 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f30feffd640 1 -- 172.21.15.31:0/430693479 learned_addr learned my addr 172.21.15.31:0/430693479 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.962+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/430693479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x56329692c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30ff7fe640 1 -- 172.21.15.31:0/430693479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 msgr2=0x56329692efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/430693479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x56329692efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30ff7fe640 1 -- 172.21.15.31:0/430693479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969219d0 msgr2=0x563296921db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/430693479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969219d0 0x563296921db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30ff7fe640 1 -- 172.21.15.31:0/430693479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329683c130 con 0x5632969222f0 2024-09-16T00:39:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30fcff9640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f30e80a3240 0x7f30e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/430693479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x56329692c670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f30f000bbf0 tx=0x7f30f00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11cb71fac022b776 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30fcff9640 1 -- 172.21.15.31:0/1292000552 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f30f80cf120 con 0x7f3100106f10 2024-09-16T00:39:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30feffd640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f30e80a3240 0x7f30e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30e5ffb640 1 -- 172.21.15.31:0/430693479 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0004030 con 0x5632969222f0 2024-09-16T00:39:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30e5ffb640 1 -- 172.21.15.31:0/430693479 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f00041d0 con 0x5632969222f0 2024-09-16T00:39:12.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.963+0000 7f30e5ffb640 1 -- 172.21.15.31:0/430693479 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0011660 con 0x5632969222f0 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 -- 172.21.15.31:0/430693479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 msgr2=0x56329692c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f30feffd640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f30e80a3240 0x7f30e80a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30f4031550 tx=0x7f30f4005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 --2- 172.21.15.31:0/430693479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x56329692c670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f30f000bbf0 tx=0x7f30f00107a0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 -- 172.21.15.31:0/430693479 shutdown_connections 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 --2- 172.21.15.31:0/430693479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x56329692efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 --2- 172.21.15.31:0/430693479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x56329692c670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 --2- 172.21.15.31:0/430693479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969219d0 0x563296921db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 -- 172.21.15.31:0/430693479 >> 172.21.15.31:0/430693479 conn(0x56329691df60 msgr2=0x56329691e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 -- 172.21.15.31:0/430693479 shutdown_connections 2024-09-16T00:39:12.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 -- 172.21.15.31:0/430693479 wait complete. 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 -- start start 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.964+0000 7f310857a740 1 Processor -- start 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f310857a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632969219d0 0x56329692c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f310857a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x563296929820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f310857a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56329692cbb0 0x563296929d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f310857a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56329692f780 con 0x56329692cbb0 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f310857a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563296842f20 con 0x5632969219d0 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f310857a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56329692f900 con 0x5632969222f0 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f310857a740 10 client.0 _ll_register_callbacks cb 0x56329680c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30feffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x563296929820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30feffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x563296929820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:48180/0 (socket says 172.21.15.31:48180) 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30feffd640 1 -- 172.21.15.31:0/882705032 learned_addr learned my addr 172.21.15.31:0/882705032 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632969219d0 0x56329692c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30fffff640 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56329692cbb0 0x563296929d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30feffd640 1 -- 172.21.15.31:0/882705032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632969219d0 msgr2=0x56329692c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30feffd640 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632969219d0 0x56329692c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30feffd640 1 -- 172.21.15.31:0/882705032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56329692cbb0 msgr2=0x563296929d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30feffd640 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56329692cbb0 0x563296929d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30feffd640 1 -- 172.21.15.31:0/882705032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632965f92b0 con 0x5632969222f0 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30fffff640 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56329692cbb0 0x563296929d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30feffd640 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x563296929820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f30f4006010 tx=0x7f30f4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a4d176ced46df52 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30deffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.965+0000 7f30deffd640 1 -- 172.21.15.31:0/882705032 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f404c070 con 0x5632969222f0 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f310857a740 1 -- 172.21.15.31:0/882705032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f00041d0 con 0x5632969222f0 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f30deffd640 1 -- 172.21.15.31:0/882705032 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f30f4005260 con 0x5632969222f0 2024-09-16T00:39:12.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f30deffd640 1 -- 172.21.15.31:0/882705032 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f30f4058050 con 0x5632969222f0 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f310857a740 10 client.24547 fetch_fsmap learned FSMap version 16 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f310857a740 1 -- 172.21.15.31:0/882705032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x56329693b150 con 0x5632969222f0 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f30deffd640 1 -- 172.21.15.31:0/882705032 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f00041d0 con 0x5632969222f0 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f30deffd640 1 -- 172.21.15.31:0/882705032 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f30f4003680 con 0x5632969222f0 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f310857a740 10 client.24547 fetch_fsmap finished waiting for FSMap version 16 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f310857a740 10 client.24547 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f310857a740 20 client.24547 populate_metadata read hostname 'smithi031' 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f310857a740 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56329693d740 0x56329693ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f310857a740 4 client.24547 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.966+0000 7f310857a740 1 -- 172.21.15.31:0/882705032 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563296895600 con 0x56329693d740 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.967+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56329693d740 0x56329693ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.967+0000 7f30deffd640 10 client.24547 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:39:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.967+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56329693d740 0x56329693ebb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97cbac5a1a144743 server_cookie=d710fc555c0c1e4f in_seq=0 out_seq=0 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f30deffd640 1 -- 172.21.15.31:0/882705032 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563296895600 con 0x56329693d740 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f30deffd640 10 client.24547 handle_command_reply: tid=0 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 client.24547 shutdown 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 -- 172.21.15.31:0/882705032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 msgr2=0x563296929820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x563296929820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f30f4006010 tx=0x7f30f4009260 comp rx=0 tx=0).stop 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 -- 172.21.15.31:0/882705032 shutdown_connections 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56329692cbb0 0x563296929d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56329693d740 0x56329693ebb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969222f0 0x563296929820 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f30deffd640 0 client.24547 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:39:12.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 --2- 172.21.15.31:0/882705032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632969219d0 0x56329692c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 -- 172.21.15.31:0/882705032 >> 172.21.15.31:0/882705032 conn(0x56329691df60 msgr2=0x563296923c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:12.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 -- 172.21.15.31:0/882705032 shutdown_connections 2024-09-16T00:39:12.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 1 -- 172.21.15.31:0/882705032 wait complete. 2024-09-16T00:39:12.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.968+0000 7f310857a740 20 client.24547 trim_cache size 0 max 16384 2024-09-16T00:39:12.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.986+0000 7f310857a740 1 Processor -- start 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.986+0000 7f310857a740 1 -- start start 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f310857a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 0x56329693ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f310857a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x56329693c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f310857a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x56329691c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f310857a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56329693cef0 con 0x5632969222f0 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f310857a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56329693d5c0 con 0x56329692cbb0 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f310857a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56329693dc90 con 0x5632969219d0 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x56329693c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x56329693c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35668/0 (socket says 172.21.15.31:35668) 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30feffd640 1 -- 172.21.15.31:0/4058140883 learned_addr learned my addr 172.21.15.31:0/4058140883 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/4058140883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 0x56329693ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30feffd640 1 -- 172.21.15.31:0/4058140883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 msgr2=0x56329693ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:12.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30feffd640 1 --2- 172.21.15.31:0/4058140883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 0x56329693ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30feffd640 1 -- 172.21.15.31:0/4058140883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 msgr2=0x56329691c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:12.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30feffd640 1 --2- 172.21.15.31:0/4058140883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x56329691c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30feffd640 1 -- 172.21.15.31:0/4058140883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563296922120 con 0x5632969222f0 2024-09-16T00:39:12.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30feffd640 1 --2- 172.21.15.31:0/4058140883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x56329693c5c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30f4003c40 tx=0x7f30f402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210be698a47156c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:12.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30e57fa640 1 -- 172.21.15.31:0/4058140883 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f404c050 con 0x5632969222f0 2024-09-16T00:39:12.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30e57fa640 1 -- 172.21.15.31:0/4058140883 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f4003cc0 con 0x5632969222f0 2024-09-16T00:39:12.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.987+0000 7f30e57fa640 1 -- 172.21.15.31:0/4058140883 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4035040 con 0x5632969222f0 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 -- 172.21.15.31:0/4058140883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 msgr2=0x56329693c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 --2- 172.21.15.31:0/4058140883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x56329693c5c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30f4003c40 tx=0x7f30f402a410 comp rx=0 tx=0).stop 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 -- 172.21.15.31:0/4058140883 shutdown_connections 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 --2- 172.21.15.31:0/4058140883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x56329691c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 --2- 172.21.15.31:0/4058140883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x56329693c5c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 --2- 172.21.15.31:0/4058140883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 0x56329693ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 -- 172.21.15.31:0/4058140883 >> 172.21.15.31:0/4058140883 conn(0x56329691df60 msgr2=0x563296924dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 -- 172.21.15.31:0/4058140883 shutdown_connections 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 -- 172.21.15.31:0/4058140883 wait complete. 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 -- start start 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.988+0000 7f310857a740 1 Processor -- start 2024-09-16T00:39:12.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f310857a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 0x563296944d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f310857a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x563296945240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f310857a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x563296940350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f310857a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56329693ea30 con 0x5632969222f0 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f310857a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56329693e360 con 0x56329692cbb0 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f310857a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56329693f100 con 0x5632969219d0 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f310857a740 10 client.0 _ll_register_callbacks cb 0x7f30f4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x563296945240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x563296945240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35682/0 (socket says 172.21.15.31:35682) 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30feffd640 1 -- 172.21.15.31:0/2411225538 learned_addr learned my addr 172.21.15.31:0/2411225538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 0x563296944d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30fffff640 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x563296940350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30feffd640 1 -- 172.21.15.31:0/2411225538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 msgr2=0x563296944d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30feffd640 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 0x563296944d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30feffd640 1 -- 172.21.15.31:0/2411225538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 msgr2=0x563296940350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30feffd640 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x563296940350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30feffd640 1 -- 172.21.15.31:0/2411225538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563296940990 con 0x5632969222f0 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 0x563296944d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30feffd640 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x563296945240 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009850 tx=0x7f30f4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c803e66fd016fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.989+0000 7f30deffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f30deffd640 1 -- 172.21.15.31:0/2411225538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4068070 con 0x5632969222f0 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f30deffd640 1 -- 172.21.15.31:0/2411225538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f30f40047c0 con 0x5632969222f0 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f310857a740 1 -- 172.21.15.31:0/2411225538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f4003cc0 con 0x5632969222f0 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f30deffd640 1 -- 172.21.15.31:0/2411225538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f30f4035070 con 0x5632969222f0 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f310857a740 10 client.14709 fetch_fsmap learned FSMap version 16 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f310857a740 1 -- 172.21.15.31:0/2411225538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5632969443c0 con 0x5632969222f0 2024-09-16T00:39:12.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f30deffd640 1 -- 172.21.15.31:0/2411225538 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f4003cc0 con 0x5632969222f0 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f30deffd640 1 -- 172.21.15.31:0/2411225538 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f30f406da30 con 0x5632969222f0 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f310857a740 10 client.14709 fetch_fsmap finished waiting for FSMap version 16 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f310857a740 10 client.14709 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f310857a740 20 client.14709 populate_metadata read hostname 'smithi031' 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f310857a740 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5632969abec0 0x5632969aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f310857a740 4 client.14709 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.990+0000 7f310857a740 1 -- 172.21.15.31:0/2411225538 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "status"}) -- 0x56329688fb10 con 0x5632969abec0 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.991+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5632969abec0 0x5632969aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.991+0000 7f30ff7fe640 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5632969abec0 0x5632969aabd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d53f6bd918a03aa server_cookie=2e2590ae8fe5d093 in_seq=0 out_seq=0 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.991+0000 7f30deffd640 10 client.14709 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.991+0000 7f30deffd640 1 -- 172.21.15.31:0/2411225538 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x56329688fb10 con 0x5632969abec0 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.991+0000 7f30deffd640 10 client.14709 handle_command_reply: tid=0 2024-09-16T00:39:12.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.991+0000 7f310857a740 1 client.14709 shutdown 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.991+0000 7f310857a740 1 -- 172.21.15.31:0/2411225538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 msgr2=0x563296945240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.991+0000 7f310857a740 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x563296945240 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009850 tx=0x7f30f4003820 comp rx=0 tx=0).stop 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f310857a740 1 -- 172.21.15.31:0/2411225538 shutdown_connections 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f310857a740 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56329692cbb0 0x563296940350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f310857a740 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632969222f0 0x563296945240 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f310857a740 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5632969abec0 0x5632969aabd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f310857a740 1 --2- 172.21.15.31:0/2411225538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632969219d0 0x563296944d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f30deffd640 0 client.14709 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f310857a740 1 -- 172.21.15.31:0/2411225538 >> 172.21.15.31:0/2411225538 conn(0x56329691df60 msgr2=0x563296927fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f310857a740 1 -- 172.21.15.31:0/2411225538 shutdown_connections 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f310857a740 1 -- 172.21.15.31:0/2411225538 wait complete. 2024-09-16T00:39:12.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f310857a740 20 client.14709 trim_cache size 0 max 16384 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "ccb601ea-73c2-11ef-bceb-c7b262605968", 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24482, 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 18.985262486, 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 16, 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 87, 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-09-16T00:39:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 29.93299923 2024-09-16T00:39:12.999 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:39:12.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 -- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f30e80a3240 msgr2=0x7f30e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:12.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f30e80a3240 0x7f30e80a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30f4031550 tx=0x7f30f4005dd0 comp rx=0 tx=0).stop 2024-09-16T00:39:12.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 -- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100106f10 msgr2=0x7f310019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:12.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100106f10 0x7f310019cd60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f30f800b970 tx=0x7f30f800be30 comp rx=0 tx=0).stop 2024-09-16T00:39:13.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 -- 172.21.15.31:0/1292000552 shutdown_connections 2024-09-16T00:39:13.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3100106f10 0x7f310019cd60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:13.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3100102790 0x7f31001989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:13.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f30e80a3240 0x7f30e80a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:13.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 --2- 172.21.15.31:0/1292000552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100101de0 0x7f3100198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:13.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.992+0000 7f31061a1640 1 -- 172.21.15.31:0/1292000552 >> 172.21.15.31:0/1292000552 conn(0x7f31000fd8c0 msgr2=0x7f31000fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:13.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.993+0000 7f31061a1640 1 -- 172.21.15.31:0/1292000552 shutdown_connections 2024-09-16T00:39:13.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:12.993+0000 7f31061a1640 1 -- 172.21.15.31:0/1292000552 wait complete. 2024-09-16T00:39:13.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:13 smithi031 ceph-mon[30956]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:13.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:13 smithi081 ceph-mon[35539]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:13.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:13 smithi102 ceph-mon[35364]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:13.795 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:39:13.795 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:39:13.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:39:14.055 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:14.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.604+0000 7fd1e7538640 1 -- 172.21.15.31:0/500419935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 msgr2=0x7fd1e01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:14.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.604+0000 7fd1e7538640 1 --2- 172.21.15.31:0/500419935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 0x7fd1e01029b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009900 tx=0x7fd1d402a480 comp rx=0 tx=0).stop 2024-09-16T00:39:14.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.604+0000 7fd1e7538640 1 -- 172.21.15.31:0/500419935 shutdown_connections 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 --2- 172.21.15.31:0/500419935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1e0102ef0 0x7fd1e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 --2- 172.21.15.31:0/500419935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 0x7fd1e01029b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 --2- 172.21.15.31:0/500419935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1e0100e20 0x7fd1e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 -- 172.21.15.31:0/500419935 >> 172.21.15.31:0/500419935 conn(0x7fd1e00fc7d0 msgr2=0x7fd1e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 -- 172.21.15.31:0/500419935 shutdown_connections 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 -- 172.21.15.31:0/500419935 wait complete. 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 Processor -- start 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 -- start start 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1e0100e20 0x7fd1e0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.605+0000 7fd1e7538640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 0x7fd1e0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e7538640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1e0102ef0 0x7fd1e019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e7538640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0110fc0 con 0x7fd1e0102570 2024-09-16T00:39:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e7538640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0110e40 con 0x7fd1e0100e20 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e7538640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0111140 con 0x7fd1e0102ef0 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e4ad3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 0x7fd1e0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e4ad3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 0x7fd1e0198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35702/0 (socket says 172.21.15.31:35702) 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e4ad3640 1 -- 172.21.15.31:0/1235221179 learned_addr learned my addr 172.21.15.31:0/1235221179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e52d4640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1e0100e20 0x7fd1e0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e5ad5640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1e0102ef0 0x7fd1e019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e4ad3640 1 -- 172.21.15.31:0/1235221179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1e0102ef0 msgr2=0x7fd1e019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e4ad3640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1e0102ef0 0x7fd1e019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e4ad3640 1 -- 172.21.15.31:0/1235221179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1e0100e20 msgr2=0x7fd1e0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e4ad3640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1e0100e20 0x7fd1e0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e4ad3640 1 -- 172.21.15.31:0/1235221179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e019d500 con 0x7fd1e0102570 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e4ad3640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 0x7fd1e0198a90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006010 tx=0x7fd1d4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1de7fc640 1 -- 172.21.15.31:0/1235221179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d403f070 con 0x7fd1e0102570 2024-09-16T00:39:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e019d790 con 0x7fd1e0102570 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1de7fc640 1 -- 172.21.15.31:0/1235221179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d4031e30 con 0x7fd1e0102570 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1de7fc640 1 -- 172.21.15.31:0/1235221179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4035030 con 0x7fd1e0102570 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.606+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e01a50d0 con 0x7fd1e0102570 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.607+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e0101200 con 0x7fd1e0102570 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.609+0000 7fd1de7fc640 1 -- 172.21.15.31:0/1235221179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd1d4042070 con 0x7fd1e0102570 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.609+0000 7fd1de7fc640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1c80a3240 0x7fd1c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.609+0000 7fd1e52d4640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1c80a3240 0x7fd1c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.609+0000 7fd1de7fc640 1 -- 172.21.15.31:0/1235221179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fd1d40f0bd0 con 0x7fd1e0102570 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.610+0000 7fd1e52d4640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1c80a3240 0x7fd1c80a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009990 tx=0x7fd1d8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.612+0000 7fd1de7fc640 1 -- 172.21.15.31:0/1235221179 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1d40bd6b0 con 0x7fd1e0102570 2024-09-16T00:39:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.823+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd1e01998b0 con 0x7fd1e0102570 2024-09-16T00:39:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.824+0000 7fd1de7fc640 1 -- 172.21.15.31:0/1235221179 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7fd1e01998b0 con 0x7fd1e0102570 2024-09-16T00:39:14.824 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T00:39:14.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1c80a3240 msgr2=0x7fd1c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:14.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1c80a3240 0x7fd1c80a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009990 tx=0x7fd1d8009110 comp rx=0 tx=0).stop 2024-09-16T00:39:14.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 msgr2=0x7fd1e0198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:14.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 0x7fd1e0198a90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006010 tx=0x7fd1d4003c80 comp rx=0 tx=0).stop 2024-09-16T00:39:14.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 shutdown_connections 2024-09-16T00:39:14.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd1c80a3240 0x7fd1c80a5700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1e0102ef0 0x7fd1e019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0102570 0x7fd1e0198a90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 --2- 172.21.15.31:0/1235221179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1e0100e20 0x7fd1e0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 >> 172.21.15.31:0/1235221179 conn(0x7fd1e00fc7d0 msgr2=0x7fd1e006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 shutdown_connections 2024-09-16T00:39:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:14.826+0000 7fd1e7538640 1 -- 172.21.15.31:0/1235221179 wait complete. 2024-09-16T00:39:15.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:15 smithi031 ceph-mon[30956]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:15.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:15 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1235221179' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:39:15.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:15 smithi081 ceph-mon[35539]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:15.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:15 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1235221179' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:39:15.504 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T00:39:15.505 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:39:15.505 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-09-16T00:39:15.505 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-16T00:39:15.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:39:15.505 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T00:39:15.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:39:15.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:39:15.505 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T00:39:15.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:15.505 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-09-16T00:39:15.524 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:39:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:39:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:39:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:39:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-09-16T00:39:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:39:15.524 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:39:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T00:39:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42299sec preferred_lft 42299sec 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-09-16T00:39:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-09-16T00:39:15.526 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:39:15.526 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:39:15.526 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:39:15.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:39:15.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-09-16T00:39:15.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-09-16T00:39:15.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:39:15.527 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:39:15.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:15 smithi102 ceph-mon[35364]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:15.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:15 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1235221179' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:39:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:39:15.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:39:15.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:15.750 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:39:15.775 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-09-16T00:39:15.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:15.776 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-09-16T00:39:15.844 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-09-16T00:39:15.844 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:39:15.844 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T00:39:15.844 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T00:39:15.844 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:39:15.845 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:39:15.845 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:39:15.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T00:39:15.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T00:39:15.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T00:39:15.845 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:39:15.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:39:16.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:39:16.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:16.162 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:16.179 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-09-16T00:39:16.234 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:39:16.234 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:39:16.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:39:16.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-16T00:39:16.234 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:39:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:39:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:39:16.354 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-16T00:39:16.355 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:39:16.355 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:39:16.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-16T00:39:16.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:39:16.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-16T00:39:16.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-16T00:39:16.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-16T00:39:16.355 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:39:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:39:16.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:39:16.487 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:39:16.487 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:39:16.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-09-16T00:39:16.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:39:16.487 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:39:16.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:39:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:39:16.567 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-16T00:39:16.567 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:39:16.567 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:39:16.583 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T00:39:16.583 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:39:16.583 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:39:16.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-09-16T00:39:16.705 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:39:16.762 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-09-16T00:39:16.762 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-09-16T00:39:16.762 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-09-16T00:39:16.762 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-09-16T00:39:16.762 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-09-16T00:39:16.762 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-09-16T00:39:16.762 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:39:16.763 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/4c046ea149e0222652b7aad243eb03b2921e7e2eccbdd555152b35e8fd845de1/merged 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/670ffc89202a37cf8e10c2e3e32db956e3f0241272836498a747742ad4a14886/merged 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/8a38e6c8b30557d571daa0c7456231da964cc886c37adb6b7e7802d528d0014e/merged 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/5a671810545ce5f2435d99b73bf9153895fb6547b7898982da4b14ee87085ec9/merged 2024-09-16T00:39:16.764 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/770db8c52afad9eabfd8d3732326799a67807ce364ecfaae8a81f6f828fdd160/merged 2024-09-16T00:39:16.765 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/9baa0760a4bde69c21c16737b2d814b42a1e18db3af1a0001f666ff79b976c4f/merged 2024-09-16T00:39:16.765 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/d08c65ed2e7992606b13663a9397551865dbd4df401c276ad098c04e6d0d86c5/merged 2024-09-16T00:39:16.765 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/642b2aa389fc309c5b3c4ccf383419fa7119dbf0e7d5790ea1dd563d94c4b574/merged 2024-09-16T00:39:16.765 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/e982e5d1e7cf5d3ae3a4340d7446c7218ed4a210f7137621d535db3d797b8bb1/merged 2024-09-16T00:39:16.765 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/88cb221504ee8bfb1e07e8546b27ea2a6ea5477417d49760bd611d96ba0ef8b1/merged 2024-09-16T00:39:16.765 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-09-16T00:39:16.765 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:39:16.765 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:39:16.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:16.765 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-09-16T00:39:16.818 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T00:39:16.818 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-16T00:39:16.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-09-16T00:39:16.886 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:39:16.943 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-09-16T00:39:16.943 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-09-16T00:39:16.943 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-09-16T00:39:16.943 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-09-16T00:39:16.943 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-09-16T00:39:16.943 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:39:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/4c046ea149e0222652b7aad243eb03b2921e7e2eccbdd555152b35e8fd845de1/merged 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/670ffc89202a37cf8e10c2e3e32db956e3f0241272836498a747742ad4a14886/merged 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/8a38e6c8b30557d571daa0c7456231da964cc886c37adb6b7e7802d528d0014e/merged 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/5a671810545ce5f2435d99b73bf9153895fb6547b7898982da4b14ee87085ec9/merged 2024-09-16T00:39:16.945 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/770db8c52afad9eabfd8d3732326799a67807ce364ecfaae8a81f6f828fdd160/merged 2024-09-16T00:39:16.946 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/9baa0760a4bde69c21c16737b2d814b42a1e18db3af1a0001f666ff79b976c4f/merged 2024-09-16T00:39:16.946 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/d08c65ed2e7992606b13663a9397551865dbd4df401c276ad098c04e6d0d86c5/merged 2024-09-16T00:39:16.946 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/642b2aa389fc309c5b3c4ccf383419fa7119dbf0e7d5790ea1dd563d94c4b574/merged 2024-09-16T00:39:16.946 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/e982e5d1e7cf5d3ae3a4340d7446c7218ed4a210f7137621d535db3d797b8bb1/merged 2024-09-16T00:39:16.946 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/88cb221504ee8bfb1e07e8546b27ea2a6ea5477417d49760bd611d96ba0ef8b1/merged 2024-09-16T00:39:16.946 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-09-16T00:39:16.946 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:39:16.946 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:39:16.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:16.946 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-09-16T00:39:16.971 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:2024-09-16T00:39:16.971+0000 7f0718726480 -1 init, newargv = 0x556dbcbe4c10 newargc=15 2024-09-16T00:39:16.971 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:ceph-fuse[72632]: starting ceph client 2024-09-16T00:39:16.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:ceph-fuse[72632]: starting fuse 2024-09-16T00:39:16.988 INFO:teuthology.orchestra.run.smithi031.stdout:120 2024-09-16T00:39:16.988 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-16T00:39:16.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-09-16T00:39:16.988 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-09-16T00:39:16.988 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-09-16T00:39:16.989 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T00:39:16.990 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T00:39:16.990 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T00:39:16.990 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-09-16T00:39:16.990 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-09-16T00:39:16.990 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-09-16T00:39:16.990 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T00:39:16.990 DEBUG:teuthology.orchestra.run.smithi031:> 2024-09-16T00:39:16.990 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.admin")) 2024-09-16T00:39:16.990 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-09-16T00:39:17.109 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.admin.72632.asok 2024-09-16T00:39:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:39:17.120 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72632.asok 2024-09-16T00:39:17.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:17.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72632.asok status 2024-09-16T00:39:17.257 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:39:17.257 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "admin", 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "72632", 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 14721, 2024-09-16T00:39:17.258 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 14721 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 544247261 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:39:17.259 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 544247261 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.14721 192.168.0.1:0/544247261", 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/544247261", 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 16, 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 87, 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-09-16T00:39:17.260 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:39:17.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:17.266 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-09-16T00:39:17.266 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:39:17.282 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-09-16T00:39:17.283 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:39:17.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:17.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:39:17.345 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:17 smithi031 ceph-mon[30956]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:17.350 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-16T00:39:17.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:39:17.466 INFO:teuthology.orchestra.run.smithi031.stdout:# file: . 2024-09-16T00:39:17.466 INFO:teuthology.orchestra.run.smithi031.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-16T00:39:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:17.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:39:17.472 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:39:17.472 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:39:17.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:39:17.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:17 smithi081 ceph-mon[35539]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:17.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:17 smithi102 ceph-mon[35364]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:17.786 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:18.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 -- 172.21.15.31:0/1358314304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb850101280 msgr2=0x7fb85010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:18.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 --2- 172.21.15.31:0/1358314304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb850101280 0x7fb85010df40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb840009900 tx=0x7fb84002a410 comp rx=0 tx=0).stop 2024-09-16T00:39:18.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 -- 172.21.15.31:0/1358314304 shutdown_connections 2024-09-16T00:39:18.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 --2- 172.21.15.31:0/1358314304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb85010e480 0x7fb850110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 --2- 172.21.15.31:0/1358314304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb850101280 0x7fb85010df40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 --2- 172.21.15.31:0/1358314304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8501008d0 0x7fb850100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 -- 172.21.15.31:0/1358314304 >> 172.21.15.31:0/1358314304 conn(0x7fb8500fc7a0 msgr2=0x7fb8500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 -- 172.21.15.31:0/1358314304 shutdown_connections 2024-09-16T00:39:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 -- 172.21.15.31:0/1358314304 wait complete. 2024-09-16T00:39:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 Processor -- start 2024-09-16T00:39:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 -- start start 2024-09-16T00:39:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8501008d0 0x7fb8501a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.271+0000 7fb8577c4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb850101280 0x7fb8501a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb8577c4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb85010e480 0x7fb85019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb8577c4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb850114260 con 0x7fb8501008d0 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb8577c4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb8501140e0 con 0x7fb850101280 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb8577c4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb8501143e0 con 0x7fb85010e480 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855560640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8501008d0 0x7fb8501a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855560640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8501008d0 0x7fb8501a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50882/0 (socket says 172.21.15.31:50882) 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855560640 1 -- 172.21.15.31:0/105712781 learned_addr learned my addr 172.21.15.31:0/105712781 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb854d5f640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb850101280 0x7fb8501a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855d61640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb85010e480 0x7fb85019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855560640 1 -- 172.21.15.31:0/105712781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb85010e480 msgr2=0x7fb85019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855560640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb85010e480 0x7fb85019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855560640 1 -- 172.21.15.31:0/105712781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb850101280 msgr2=0x7fb8501a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855560640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb850101280 0x7fb8501a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855560640 1 -- 172.21.15.31:0/105712781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85019aee0 con 0x7fb8501008d0 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb855560640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8501008d0 0x7fb8501a05a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b4a0 tx=0x7fb84c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb8467fc640 1 -- 172.21.15.31:0/105712781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c014070 con 0x7fb8501008d0 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb8467fc640 1 -- 172.21.15.31:0/105712781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84c010af0 con 0x7fb8501008d0 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85019b1d0 con 0x7fb8501008d0 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb8467fc640 1 -- 172.21.15.31:0/105712781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c017d10 con 0x7fb8501008d0 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.272+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8501a7520 con 0x7fb8501008d0 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.274+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb85006b070 con 0x7fb8501008d0 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.275+0000 7fb8467fc640 1 -- 172.21.15.31:0/105712781 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb84c017070 con 0x7fb8501008d0 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.276+0000 7fb8467fc640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb83c0a32c0 0x7fb83c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.276+0000 7fb854d5f640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb83c0a32c0 0x7fb83c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.276+0000 7fb8467fc640 1 -- 172.21.15.31:0/105712781 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fb84c0cee40 con 0x7fb8501008d0 2024-09-16T00:39:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.276+0000 7fb854d5f640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb83c0a32c0 0x7fb83c0a5780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb840009230 tx=0x7fb8400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.280+0000 7fb8467fc640 1 -- 172.21.15.31:0/105712781 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb84c09b920 con 0x7fb8501008d0 2024-09-16T00:39:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.471+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb850109610 con 0x7fb8501008d0 2024-09-16T00:39:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.471+0000 7fb8467fc640 1 -- 172.21.15.31:0/105712781 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+588 (secure 0 0 0) 0x7fb84c09b220 con 0x7fb8501008d0 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2779988710 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/4219649043 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/945234876 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3876592548 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/4219649043 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/945234876 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/876405889 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2258150314 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/880818822 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2222283837 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:18.473 INFO:teuthology.orchestra.run.smithi031.stderr:listed 10 entries 2024-09-16T00:39:18.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb83c0a32c0 msgr2=0x7fb83c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:18.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb83c0a32c0 0x7fb83c0a5780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb840009230 tx=0x7fb8400058d0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8501008d0 msgr2=0x7fb8501a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8501008d0 0x7fb8501a05a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b4a0 tx=0x7fb84c00b960 comp rx=0 tx=0).stop 2024-09-16T00:39:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 shutdown_connections 2024-09-16T00:39:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb83c0a32c0 0x7fb83c0a5780 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb85010e480 0x7fb85019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb850101280 0x7fb8501a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 --2- 172.21.15.31:0/105712781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8501008d0 0x7fb8501a05a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.473+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 >> 172.21.15.31:0/105712781 conn(0x7fb8500fc7a0 msgr2=0x7fb8500feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.474+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 shutdown_connections 2024-09-16T00:39:18.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:18.474+0000 7fb8577c4640 1 -- 172.21.15.31:0/105712781 wait complete. 2024-09-16T00:39:19.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:19 smithi031 ceph-mon[30956]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:19.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:19 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/105712781' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:39:19.269 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:39:19.269 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:39:19.289 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:39:19.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:19 smithi081 ceph-mon[35539]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:19.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:19 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/105712781' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:39:19.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:19 smithi102 ceph-mon[35364]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:19.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:19 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/105712781' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:39:19.599 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.037+0000 7f03db318640 1 -- 172.21.15.31:0/1350635212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 msgr2=0x7f03d4100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.037+0000 7f03db318640 1 --2- 172.21.15.31:0/1350635212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 0x7f03d4100cb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009900 tx=0x7f03d002a410 comp rx=0 tx=0).stop 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.037+0000 7f03db318640 1 -- 172.21.15.31:0/1350635212 shutdown_connections 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.037+0000 7f03db318640 1 --2- 172.21.15.31:0/1350635212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d410e480 0x7f03d4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.037+0000 7f03db318640 1 --2- 172.21.15.31:0/1350635212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03d4101280 0x7f03d410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.037+0000 7f03db318640 1 --2- 172.21.15.31:0/1350635212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 0x7f03d4100cb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.037+0000 7f03db318640 1 -- 172.21.15.31:0/1350635212 >> 172.21.15.31:0/1350635212 conn(0x7f03d40fc7a0 msgr2=0x7f03d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.037+0000 7f03db318640 1 -- 172.21.15.31:0/1350635212 shutdown_connections 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.037+0000 7f03db318640 1 -- 172.21.15.31:0/1350635212 wait complete. 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03db318640 1 Processor -- start 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03db318640 1 -- start start 2024-09-16T00:39:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03db318640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 0x7f03d419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03db318640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4101280 0x7f03d419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03db318640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03d410e480 0x7f03d419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03db318640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f03d41141b0 con 0x7f03d41008d0 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03db318640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f03d4114030 con 0x7f03d4101280 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03db318640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f03d4114330 con 0x7f03d410e480 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d90b4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 0x7f03d419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d90b4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 0x7f03d419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50894/0 (socket says 172.21.15.31:50894) 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d90b4640 1 -- 172.21.15.31:0/2190446988 learned_addr learned my addr 172.21.15.31:0/2190446988 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d98b5640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03d410e480 0x7f03d419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d88b3640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4101280 0x7f03d419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d90b4640 1 -- 172.21.15.31:0/2190446988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03d410e480 msgr2=0x7f03d419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d90b4640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03d410e480 0x7f03d419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d90b4640 1 -- 172.21.15.31:0/2190446988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4101280 msgr2=0x7f03d419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d90b4640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4101280 0x7f03d419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d90b4640 1 -- 172.21.15.31:0/2190446988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d419f640 con 0x7f03d41008d0 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.038+0000 7f03d88b3640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4101280 0x7f03d419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.039+0000 7f03d90b4640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 0x7f03d419a690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f03d0006010 tx=0x7f03d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.039+0000 7f03ca7fc640 1 -- 172.21.15.31:0/2190446988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d003f030 con 0x7f03d41008d0 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.039+0000 7f03ca7fc640 1 -- 172.21.15.31:0/2190446988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d0003ce0 con 0x7f03d41008d0 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.039+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d419f8d0 con 0x7f03d41008d0 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.039+0000 7f03ca7fc640 1 -- 172.21.15.31:0/2190446988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0035070 con 0x7f03d41008d0 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.039+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d41a7210 con 0x7f03d41008d0 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.040+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d406b070 con 0x7f03d41008d0 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.041+0000 7f03ca7fc640 1 -- 172.21.15.31:0/2190446988 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f03d0042030 con 0x7f03d41008d0 2024-09-16T00:39:20.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.042+0000 7f03ca7fc640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f03c00a3240 0x7f03c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.042+0000 7f03d88b3640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f03c00a3240 0x7f03c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.042+0000 7f03ca7fc640 1 -- 172.21.15.31:0/2190446988 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f03d00f01e0 con 0x7f03d41008d0 2024-09-16T00:39:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.042+0000 7f03d88b3640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f03c00a3240 0x7f03c00a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f03d410ed80 tx=0x7f03c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.046+0000 7f03ca7fc640 1 -- 172.21.15.31:0/2190446988 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03d00bcd80 con 0x7f03d41008d0 2024-09-16T00:39:20.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.234+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f03d4109610 con 0x7f03d41008d0 2024-09-16T00:39:20.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.235+0000 7f03ca7fc640 1 -- 172.21.15.31:0/2190446988 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+588 (secure 0 0 0) 0x7f03d00bc680 con 0x7f03d41008d0 2024-09-16T00:39:20.235 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2779988710 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:20.235 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/4219649043 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:20.235 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/945234876 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:20.236 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3876592548 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:20.236 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/4219649043 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:20.236 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/945234876 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:20.236 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/876405889 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:20.236 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2258150314 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:20.236 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/880818822 2024-09-17T00:31:57.569184+0000 2024-09-16T00:39:20.236 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2222283837 2024-09-17T00:31:33.279333+0000 2024-09-16T00:39:20.236 INFO:teuthology.orchestra.run.smithi031.stderr:listed 10 entries 2024-09-16T00:39:20.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f03c00a3240 msgr2=0x7f03c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:20.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f03c00a3240 0x7f03c00a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f03d410ed80 tx=0x7f03c4005f90 comp rx=0 tx=0).stop 2024-09-16T00:39:20.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 msgr2=0x7f03d419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 0x7f03d419a690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f03d0006010 tx=0x7f03d00042f0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 shutdown_connections 2024-09-16T00:39:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f03c00a3240 0x7f03c00a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03d410e480 0x7f03d419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03d4101280 0x7f03d419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 --2- 172.21.15.31:0/2190446988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03d41008d0 0x7f03d419a690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 >> 172.21.15.31:0/2190446988 conn(0x7f03d40fc7a0 msgr2=0x7f03d40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 shutdown_connections 2024-09-16T00:39:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:20.237+0000 7f03db318640 1 -- 172.21.15.31:0/2190446988 wait complete. 2024-09-16T00:39:20.985 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-09-16T00:39:20.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:20.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:39:21.015 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:ceph-fuse[72632]: fuse finished with error 0 and tester_r 0 2024-09-16T00:39:21.016 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:39:21.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:21 smithi031 ceph-mon[30956]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:21.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:21 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2190446988' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:39:21.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:21 smithi081 ceph-mon[35539]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:21.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:21 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2190446988' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:39:21.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:21 smithi102 ceph-mon[35364]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:39:21.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:21 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2190446988' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:39:23.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:23 smithi031 ceph-mon[30956]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:39:23.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:23 smithi081 ceph-mon[35539]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:39:23.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:23 smithi102 ceph-mon[35364]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:39:25.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:25 smithi031 ceph-mon[30956]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:39:25.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:25 smithi081 ceph-mon[35539]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:39:25.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:25 smithi102 ceph-mon[35364]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:39:27.118 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:39:27.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:39:27.119 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-16T00:39:27.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:39:27.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:27 smithi031 ceph-mon[30956]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:27.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:27 smithi081 ceph-mon[35539]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:27.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:27 smithi102 ceph-mon[35364]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:27.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.939+0000 7fa41307f640 1 -- 172.21.15.31:0/3279304713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1082a0 msgr2=0x7fa40c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.939+0000 7fa41307f640 1 --2- 172.21.15.31:0/3279304713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1082a0 0x7fa40c10ec40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa404009cd0 tx=0x7fa40402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.939+0000 7fa41307f640 1 -- 172.21.15.31:0/3279304713 shutdown_connections 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.939+0000 7fa41307f640 1 --2- 172.21.15.31:0/3279304713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1082a0 0x7fa40c10ec40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.939+0000 7fa41307f640 1 --2- 172.21.15.31:0/3279304713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa40c1036e0 0x7fa40c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.939+0000 7fa41307f640 1 --2- 172.21.15.31:0/3279304713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa40c102d30 0x7fa40c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.939+0000 7fa41307f640 1 -- 172.21.15.31:0/3279304713 >> 172.21.15.31:0/3279304713 conn(0x7fa40c0fc7d0 msgr2=0x7fa40c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.939+0000 7fa41307f640 1 -- 172.21.15.31:0/3279304713 shutdown_connections 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.939+0000 7fa41307f640 1 -- 172.21.15.31:0/3279304713 wait complete. 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41307f640 1 Processor -- start 2024-09-16T00:39:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41307f640 1 -- start start 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41307f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa40c102d30 0x7fa40c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41307f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1036e0 0x7fa40c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41307f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa40c1082a0 0x7fa40c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41307f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1120a0 con 0x7fa40c1036e0 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41307f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c111f20 con 0x7fa40c102d30 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41307f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c112220 con 0x7fa40c1082a0 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1036e0 0x7fa40c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1036e0 0x7fa40c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46628/0 (socket says 172.21.15.31:46628) 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa40bfff640 1 -- 172.21.15.31:0/4250760792 learned_addr learned my addr 172.21.15.31:0/4250760792 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa410e1b640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa40c102d30 0x7fa40c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa40bfff640 1 -- 172.21.15.31:0/4250760792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa40c1082a0 msgr2=0x7fa40c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:27.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41161c640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa40c1082a0 0x7fa40c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa40bfff640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa40c1082a0 0x7fa40c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa40bfff640 1 -- 172.21.15.31:0/4250760792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa40c102d30 msgr2=0x7fa40c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa40bfff640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa40c102d30 0x7fa40c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa40bfff640 1 -- 172.21.15.31:0/4250760792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c1a69f0 con 0x7fa40c1036e0 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa41161c640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa40c1082a0 0x7fa40c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.940+0000 7fa410e1b640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa40c102d30 0x7fa40c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.941+0000 7fa40bfff640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1036e0 0x7fa40c100ed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9f0 tx=0x7fa3fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.941+0000 7fa409ffb640 1 -- 172.21.15.31:0/4250760792 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00ccc0 con 0x7fa40c1036e0 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.941+0000 7fa409ffb640 1 -- 172.21.15.31:0/4250760792 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3fc010910 con 0x7fa40c1036e0 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.941+0000 7fa409ffb640 1 -- 172.21.15.31:0/4250760792 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc018920 con 0x7fa40c1036e0 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.941+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1a6ce0 con 0x7fa40c1036e0 2024-09-16T00:39:27.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.941+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1a7170 con 0x7fa40c1036e0 2024-09-16T00:39:27.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.942+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c103110 con 0x7fa40c1036e0 2024-09-16T00:39:27.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.943+0000 7fa409ffb640 1 -- 172.21.15.31:0/4250760792 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa3fc010ac0 con 0x7fa40c1036e0 2024-09-16T00:39:27.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.944+0000 7fa409ffb640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa3f80a32c0 0x7fa3f80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:27.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.944+0000 7fa409ffb640 1 -- 172.21.15.31:0/4250760792 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fa3fc0d0220 con 0x7fa40c1036e0 2024-09-16T00:39:27.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.944+0000 7fa410e1b640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa3f80a32c0 0x7fa3f80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:27.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.945+0000 7fa410e1b640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa3f80a32c0 0x7fa3f80a5780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009c20 tx=0x7fa3f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:27.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:27.947+0000 7fa409ffb640 1 -- 172.21.15.31:0/4250760792 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3fc09cda0 con 0x7fa40c1036e0 2024-09-16T00:39:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.126+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa40c101ec0 con 0x7fa40c1036e0 2024-09-16T00:39:28.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.127+0000 7fa409ffb640 1 -- 172.21.15.31:0/4250760792 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24150 (secure 0 0 0) 0x7fa3fc09c6a0 con 0x7fa40c1036e0 2024-09-16T00:39:28.129 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:28.129 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:39:02.716519+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:39:28.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa3f80a32c0 msgr2=0x7fa3f80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:28.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa3f80a32c0 0x7fa3f80a5780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009c20 tx=0x7fa3f4005f90 comp rx=0 tx=0).stop 2024-09-16T00:39:28.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1036e0 msgr2=0x7fa40c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:28.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1036e0 0x7fa40c100ed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9f0 tx=0x7fa3fc00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 shutdown_connections 2024-09-16T00:39:28.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa3f80a32c0 0x7fa3f80a5780 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa40c1082a0 0x7fa40c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa40c1036e0 0x7fa40c100ed0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 --2- 172.21.15.31:0/4250760792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa40c102d30 0x7fa40c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 >> 172.21.15.31:0/4250760792 conn(0x7fa40c0fc7d0 msgr2=0x7fa40c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 shutdown_connections 2024-09-16T00:39:28.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:28.130+0000 7fa41307f640 1 -- 172.21.15.31:0/4250760792 wait complete. 2024-09-16T00:39:28.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:39:29.219 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:29.341 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:29 smithi031 ceph-mon[30956]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:29.341 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:29 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4250760792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:29.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:29 smithi081 ceph-mon[35539]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:29.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:29 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4250760792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:29.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:29 smithi102 ceph-mon[35364]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:29.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:29 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4250760792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:29.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.712+0000 7f4424c3c640 1 -- 172.21.15.31:0/2480607469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 msgr2=0x7f4420103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.712+0000 7f4424c3c640 1 --2- 172.21.15.31:0/2480607469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 0x7f4420103120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4410009900 tx=0x7f441002a480 comp rx=0 tx=0).stop 2024-09-16T00:39:29.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.712+0000 7f4424c3c640 1 -- 172.21.15.31:0/2480607469 shutdown_connections 2024-09-16T00:39:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.712+0000 7f4424c3c640 1 --2- 172.21.15.31:0/2480607469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f442010c170 0x7f442010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.712+0000 7f4424c3c640 1 --2- 172.21.15.31:0/2480607469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4420103660 0x7f442010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.713+0000 7f4424c3c640 1 --2- 172.21.15.31:0/2480607469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 0x7f4420103120 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.713+0000 7f4424c3c640 1 -- 172.21.15.31:0/2480607469 >> 172.21.15.31:0/2480607469 conn(0x7f44200fc7a0 msgr2=0x7f44200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.713+0000 7f4424c3c640 1 -- 172.21.15.31:0/2480607469 shutdown_connections 2024-09-16T00:39:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.713+0000 7f4424c3c640 1 -- 172.21.15.31:0/2480607469 wait complete. 2024-09-16T00:39:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.713+0000 7f4424c3c640 1 Processor -- start 2024-09-16T00:39:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.713+0000 7f4424c3c640 1 -- start start 2024-09-16T00:39:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f4424c3c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 0x7f4420198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f4424c3c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4420103660 0x7f4420198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f4424c3c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f442010c170 0x7f442019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f4424c3c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4420112030 con 0x7f4420102d40 2024-09-16T00:39:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f4424c3c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4420111eb0 con 0x7f442010c170 2024-09-16T00:39:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f4424c3c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f44201121b0 con 0x7f4420103660 2024-09-16T00:39:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 0x7f4420198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 0x7f4420198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46642/0 (socket says 172.21.15.31:46642) 2024-09-16T00:39:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441e59c640 1 -- 172.21.15.31:0/734311105 learned_addr learned my addr 172.21.15.31:0/734311105 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441dd9b640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4420103660 0x7f4420198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441ed9d640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f442010c170 0x7f442019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441e59c640 1 -- 172.21.15.31:0/734311105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4420103660 msgr2=0x7f4420198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441e59c640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4420103660 0x7f4420198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441e59c640 1 -- 172.21.15.31:0/734311105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f442010c170 msgr2=0x7f442019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441e59c640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f442010c170 0x7f442019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441e59c640 1 -- 172.21.15.31:0/734311105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442019d640 con 0x7f4420102d40 2024-09-16T00:39:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441dd9b640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4420103660 0x7f4420198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f441e59c640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 0x7f4420198690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4410005ab0 tx=0x7f44100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.714+0000 7f440f7fe640 1 -- 172.21.15.31:0/734311105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441003f030 con 0x7f4420102d40 2024-09-16T00:39:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.715+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442019d8d0 con 0x7f4420102d40 2024-09-16T00:39:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.715+0000 7f440f7fe640 1 -- 172.21.15.31:0/734311105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4410003ce0 con 0x7f4420102d40 2024-09-16T00:39:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.715+0000 7f440f7fe640 1 -- 172.21.15.31:0/734311105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410035070 con 0x7f4420102d40 2024-09-16T00:39:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.715+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44201a5210 con 0x7f4420102d40 2024-09-16T00:39:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.715+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442006b070 con 0x7f4420102d40 2024-09-16T00:39:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.717+0000 7f440f7fe640 1 -- 172.21.15.31:0/734311105 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4410042030 con 0x7f4420102d40 2024-09-16T00:39:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.717+0000 7f440f7fe640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44080a3240 0x7f44080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:29.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.717+0000 7f440f7fe640 1 -- 172.21.15.31:0/734311105 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f44100f0770 con 0x7f4420102d40 2024-09-16T00:39:29.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.717+0000 7f441dd9b640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44080a3240 0x7f44080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:29.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.718+0000 7f441dd9b640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44080a3240 0x7f44080a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f44200fee70 tx=0x7f4414008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:29.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.721+0000 7f440f7fe640 1 -- 172.21.15.31:0/734311105 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44100bd250 con 0x7f4420102d40 2024-09-16T00:39:29.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.897+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44201031a0 con 0x7f4420102d40 2024-09-16T00:39:29.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.898+0000 7f440f7fe640 1 -- 172.21.15.31:0/734311105 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24150 (secure 0 0 0) 0x7f44100bcb50 con 0x7f4420102d40 2024-09-16T00:39:29.901 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:29.901 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:39:02.716519+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:39:29.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44080a3240 msgr2=0x7f44080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44080a3240 0x7f44080a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f44200fee70 tx=0x7f4414008040 comp rx=0 tx=0).stop 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 msgr2=0x7f4420198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 0x7f4420198690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4410005ab0 tx=0x7f44100042f0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 shutdown_connections 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f442010c170 0x7f442019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44080a3240 0x7f44080a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4420103660 0x7f4420198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 --2- 172.21.15.31:0/734311105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4420102d40 0x7f4420198690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 >> 172.21.15.31:0/734311105 conn(0x7f44200fc7a0 msgr2=0x7f44200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 shutdown_connections 2024-09-16T00:39:29.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:29.900+0000 7f4424c3c640 1 -- 172.21.15.31:0/734311105 wait complete. 2024-09-16T00:39:30.209 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:30 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/734311105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:30.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:30 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/734311105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:30.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:30 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/734311105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:30.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-16T00:39:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:31.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:31 smithi031 ceph-mon[30956]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.339+0000 7ffa87f3a640 1 -- 172.21.15.31:0/3976800630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801008c0 msgr2=0x7ffa80100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.339+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/3976800630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801008c0 0x7ffa80100ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009ca0 tx=0x7ffa7002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.339+0000 7ffa87f3a640 1 -- 172.21.15.31:0/3976800630 shutdown_connections 2024-09-16T00:39:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.339+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/3976800630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa80105ec0 0x7ffa80106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.339+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/3976800630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa801011e0 0x7ffa80105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.339+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/3976800630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801008c0 0x7ffa80100ca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.339+0000 7ffa87f3a640 1 -- 172.21.15.31:0/3976800630 >> 172.21.15.31:0/3976800630 conn(0x7ffa800fc7d0 msgr2=0x7ffa800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.339+0000 7ffa87f3a640 1 -- 172.21.15.31:0/3976800630 shutdown_connections 2024-09-16T00:39:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.339+0000 7ffa87f3a640 1 -- 172.21.15.31:0/3976800630 wait complete. 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa87f3a640 1 Processor -- start 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa87f3a640 1 -- start start 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa87f3a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa801008c0 0x7ffa8019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa87f3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801011e0 0x7ffa8019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa87f3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa80105ec0 0x7ffa801a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa87f3a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80118610 con 0x7ffa801011e0 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa87f3a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80118490 con 0x7ffa801008c0 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa87f3a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80118790 con 0x7ffa80105ec0 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa854d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801011e0 0x7ffa8019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa854d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801011e0 0x7ffa8019eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46660/0 (socket says 172.21.15.31:46660) 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa854d5640 1 -- 172.21.15.31:0/5328535 learned_addr learned my addr 172.21.15.31:0/5328535 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa85cd6640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa801008c0 0x7ffa8019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa864d7640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa80105ec0 0x7ffa801a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.340+0000 7ffa854d5640 1 -- 172.21.15.31:0/5328535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa80105ec0 msgr2=0x7ffa801a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa854d5640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa80105ec0 0x7ffa801a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa854d5640 1 -- 172.21.15.31:0/5328535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa801008c0 msgr2=0x7ffa8019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa854d5640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa801008c0 0x7ffa8019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa854d5640 1 -- 172.21.15.31:0/5328535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa801a3ae0 con 0x7ffa801011e0 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa864d7640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa80105ec0 0x7ffa801a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa854d5640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801011e0 0x7ffa8019eff0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c920 tx=0x7ffa7400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa7effd640 1 -- 172.21.15.31:0/5328535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74007c70 con 0x7ffa801011e0 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa801a3dd0 con 0x7ffa801011e0 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa7effd640 1 -- 172.21.15.31:0/5328535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa74007e10 con 0x7ffa801011e0 2024-09-16T00:39:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa7effd640 1 -- 172.21.15.31:0/5328535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74018890 con 0x7ffa801011e0 2024-09-16T00:39:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.341+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa801ab660 con 0x7ffa801011e0 2024-09-16T00:39:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.342+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa8006a850 con 0x7ffa801011e0 2024-09-16T00:39:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.343+0000 7ffa7effd640 1 -- 172.21.15.31:0/5328535 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ffa74018a50 con 0x7ffa801011e0 2024-09-16T00:39:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.344+0000 7ffa7effd640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ffa680a3240 0x7ffa680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.344+0000 7ffa85cd6640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ffa680a3240 0x7ffa680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.344+0000 7ffa7effd640 1 -- 172.21.15.31:0/5328535 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7ffa740cfff0 con 0x7ffa801011e0 2024-09-16T00:39:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.345+0000 7ffa85cd6640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ffa680a3240 0x7ffa680a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffa70000f80 tx=0x7ffa70031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.347+0000 7ffa7effd640 1 -- 172.21.15.31:0/5328535 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa74014030 con 0x7ffa801011e0 2024-09-16T00:39:31.507 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:31 smithi081 ceph-mon[35539]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:31.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:31 smithi102 ceph-mon[35364]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:31.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:31.562+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffa80105980 con 0x7ffa801011e0 2024-09-16T00:39:32.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.122+0000 7ffa7effd640 1 -- 172.21.15.31:0/5328535 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7ffa7409cb60 con 0x7ffa801011e0 2024-09-16T00:39:32.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ffa680a3240 msgr2=0x7ffa680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ffa680a3240 0x7ffa680a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffa70000f80 tx=0x7ffa70031000 comp rx=0 tx=0).stop 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801011e0 msgr2=0x7ffa8019eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801011e0 0x7ffa8019eff0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c920 tx=0x7ffa7400cde0 comp rx=0 tx=0).stop 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 shutdown_connections 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ffa680a3240 0x7ffa680a5700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa80105ec0 0x7ffa801a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa801011e0 0x7ffa8019eff0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 --2- 172.21.15.31:0/5328535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa801008c0 0x7ffa8019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 >> 172.21.15.31:0/5328535 conn(0x7ffa800fc7d0 msgr2=0x7ffa8010f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 shutdown_connections 2024-09-16T00:39:32.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:32.125+0000 7ffa87f3a640 1 -- 172.21.15.31:0/5328535 wait complete. 2024-09-16T00:39:32.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:32 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/5328535' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:39:32.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:32 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/5328535' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:39:32.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:32 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/5328535' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:39:32.961 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-16T00:39:32.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-16T00:39:33.229 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:33.350 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:33 smithi031 ceph-mon[30956]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:33.351 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:33 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/5328535' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:39:33.351 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:33 smithi031 ceph-mon[30956]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:39:33.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:33 smithi081 ceph-mon[35539]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:33.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:33 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/5328535' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:39:33.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:33 smithi081 ceph-mon[35539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:39:33.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:33 smithi102 ceph-mon[35364]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:39:33.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:33 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/5328535' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:39:33.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:33 smithi102 ceph-mon[35364]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:39:33.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 -- 172.21.15.31:0/208280668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 msgr2=0x7f26681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 --2- 172.21.15.31:0/208280668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 0x7f26681067e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2658009900 tx=0x7f265802a410 comp rx=0 tx=0).stop 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 -- 172.21.15.31:0/208280668 shutdown_connections 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 --2- 172.21.15.31:0/208280668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2668106f10 0x7f266810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 --2- 172.21.15.31:0/208280668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 0x7f26681067e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 --2- 172.21.15.31:0/208280668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2668101de0 0x7f26681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 -- 172.21.15.31:0/208280668 >> 172.21.15.31:0/208280668 conn(0x7f26680fd8c0 msgr2=0x7f26680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 -- 172.21.15.31:0/208280668 shutdown_connections 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 -- 172.21.15.31:0/208280668 wait complete. 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 Processor -- start 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.739+0000 7f26706f2640 1 -- start start 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f26706f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2668101de0 0x7f2668198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f26706f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 0x7f26681989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f26706f2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2668106f10 0x7f266819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f26706f2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2668111f70 con 0x7f2668102790 2024-09-16T00:39:33.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f26706f2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2668111df0 con 0x7f2668101de0 2024-09-16T00:39:33.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f26706f2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f26681120f0 con 0x7f2668106f10 2024-09-16T00:39:33.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266dc8d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 0x7f26681989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266dc8d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 0x7f26681989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46668/0 (socket says 172.21.15.31:46668) 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266e48e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2668101de0 0x7f2668198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266dc8d640 1 -- 172.21.15.31:0/919697591 learned_addr learned my addr 172.21.15.31:0/919697591 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266ec8f640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2668106f10 0x7f266819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266dc8d640 1 -- 172.21.15.31:0/919697591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2668106f10 msgr2=0x7f266819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266dc8d640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2668106f10 0x7f266819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266dc8d640 1 -- 172.21.15.31:0/919697591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2668101de0 msgr2=0x7f2668198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266dc8d640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2668101de0 0x7f2668198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266dc8d640 1 -- 172.21.15.31:0/919697591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266819d440 con 0x7f2668102790 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.740+0000 7f266dc8d640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 0x7f26681989d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2658006010 tx=0x7f26580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.741+0000 7f265f7fe640 1 -- 172.21.15.31:0/919697591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265803f030 con 0x7f2668102790 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.741+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266819d6d0 con 0x7f2668102790 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.741+0000 7f265f7fe640 1 -- 172.21.15.31:0/919697591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2658003ce0 con 0x7f2668102790 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.741+0000 7f265f7fe640 1 -- 172.21.15.31:0/919697591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658035070 con 0x7f2668102790 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.741+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26681a5010 con 0x7f2668102790 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.742+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26681021c0 con 0x7f2668102790 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.743+0000 7f265f7fe640 1 -- 172.21.15.31:0/919697591 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2658042030 con 0x7f2668102790 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.744+0000 7f265f7fe640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26540a3240 0x7f26540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.744+0000 7f266e48e640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26540a3240 0x7f26540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.744+0000 7f265f7fe640 1 -- 172.21.15.31:0/919697591 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f26580f0220 con 0x7f2668102790 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.744+0000 7f266e48e640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26540a3240 0x7f26540a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f266400bbe0 tx=0x7f266400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:33.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.747+0000 7f265f7fe640 1 -- 172.21.15.31:0/919697591 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26580bc810 con 0x7f2668102790 2024-09-16T00:39:33.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:33.959+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f26681997f0 con 0x7f2668102790 2024-09-16T00:39:34.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.129+0000 7f265f7fe640 1 -- 172.21.15.31:0/919697591 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f26681997f0 con 0x7f2668102790 2024-09-16T00:39:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26540a3240 msgr2=0x7f26540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26540a3240 0x7f26540a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f266400bbe0 tx=0x7f266400b390 comp rx=0 tx=0).stop 2024-09-16T00:39:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 msgr2=0x7f26681989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 0x7f26681989d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2658006010 tx=0x7f26580042f0 comp rx=0 tx=0).stop 2024-09-16T00:39:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 shutdown_connections 2024-09-16T00:39:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26540a3240 0x7f26540a5700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2668106f10 0x7f266819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:34.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2668102790 0x7f26681989d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:34.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 --2- 172.21.15.31:0/919697591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2668101de0 0x7f2668198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:34.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 >> 172.21.15.31:0/919697591 conn(0x7f26680fd8c0 msgr2=0x7f26680fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:34.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 shutdown_connections 2024-09-16T00:39:34.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:34.131+0000 7f26706f2640 1 -- 172.21.15.31:0/919697591 wait complete. 2024-09-16T00:39:34.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:34 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/919697591' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:39:34.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:34 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/919697591' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:39:34.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:34 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/919697591' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:39:34.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-16T00:39:35.257 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:35.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:35 smithi031 ceph-mon[30956]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-16T00:39:35.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:35 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/919697591' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:39:35.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:35 smithi031 ceph-mon[30956]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:39:35.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:35 smithi031 ceph-mon[30956]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-16T00:39:35.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:35 smithi081 ceph-mon[35539]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-16T00:39:35.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:35 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/919697591' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:39:35.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:35 smithi081 ceph-mon[35539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:39:35.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:35 smithi081 ceph-mon[35539]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-16T00:39:35.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:35 smithi102 ceph-mon[35364]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-16T00:39:35.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:35 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/919697591' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:39:35.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:35 smithi102 ceph-mon[35364]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:39:35.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:35 smithi102 ceph-mon[35364]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-16T00:39:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.733+0000 7f28462d8640 1 -- 172.21.15.31:0/543856067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 msgr2=0x7f284010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.733+0000 7f28462d8640 1 --2- 172.21.15.31:0/543856067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 0x7f284010eaf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2838009cd0 tx=0x7f283802a8f0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.733+0000 7f28462d8640 1 -- 172.21.15.31:0/543856067 shutdown_connections 2024-09-16T00:39:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.733+0000 7f28462d8640 1 --2- 172.21.15.31:0/543856067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 0x7f284010eaf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.733+0000 7f28462d8640 1 --2- 172.21.15.31:0/543856067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2840100ca0 0x7f2840106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.733+0000 7f28462d8640 1 --2- 172.21.15.31:0/543856067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28401002f0 0x7f28401006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.733+0000 7f28462d8640 1 -- 172.21.15.31:0/543856067 >> 172.21.15.31:0/543856067 conn(0x7f2840077210 msgr2=0x7f28400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.733+0000 7f28462d8640 1 -- 172.21.15.31:0/543856067 shutdown_connections 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.733+0000 7f28462d8640 1 -- 172.21.15.31:0/543856067 wait complete. 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f28462d8640 1 Processor -- start 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f28462d8640 1 -- start start 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f28462d8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28401002f0 0x7f2840198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f28462d8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2840100ca0 0x7f28401989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f28462d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 0x7f284019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f28462d8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2840112000 con 0x7f2840106f50 2024-09-16T00:39:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f28462d8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2840111e80 con 0x7f2840100ca0 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f28462d8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2840112180 con 0x7f28401002f0 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f2844875640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 0x7f284019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f2844875640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 0x7f284019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46686/0 (socket says 172.21.15.31:46686) 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.734+0000 7f2844875640 1 -- 172.21.15.31:0/4150679331 learned_addr learned my addr 172.21.15.31:0/4150679331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f283ffff640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28401002f0 0x7f2840198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f2844875640 1 -- 172.21.15.31:0/4150679331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28401002f0 msgr2=0x7f2840198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f2844875640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28401002f0 0x7f2840198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f2844875640 1 -- 172.21.15.31:0/4150679331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2840100ca0 msgr2=0x7f28401989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f283f7fe640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2840100ca0 0x7f28401989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f2844875640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2840100ca0 0x7f28401989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f2844875640 1 -- 172.21.15.31:0/4150679331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284019d430 con 0x7f2840106f50 2024-09-16T00:39:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f283ffff640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28401002f0 0x7f2840198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f283f7fe640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2840100ca0 0x7f28401989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f2844875640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 0x7f284019cd50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2838009ca0 tx=0x7f28380039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f283d7fa640 1 -- 172.21.15.31:0/4150679331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838004040 con 0x7f2840106f50 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284019d6c0 con 0x7f2840106f50 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f283d7fa640 1 -- 172.21.15.31:0/4150679331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2838040070 con 0x7f2840106f50 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f283d7fa640 1 -- 172.21.15.31:0/4150679331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838045da0 con 0x7f2840106f50 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.735+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28401a5000 con 0x7f2840106f50 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.736+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28401006d0 con 0x7f2840106f50 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.738+0000 7f283d7fa640 1 -- 172.21.15.31:0/4150679331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2838004b40 con 0x7f2840106f50 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.738+0000 7f283d7fa640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28280a32c0 0x7f28280a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.738+0000 7f283d7fa640 1 -- 172.21.15.31:0/4150679331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f28380f1170 con 0x7f2840106f50 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.738+0000 7f283ffff640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28280a32c0 0x7f28280a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.739+0000 7f283ffff640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28280a32c0 0x7f28280a5780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f283000baa0 tx=0x7f2830005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:35.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.742+0000 7f283d7fa640 1 -- 172.21.15.31:0/4150679331 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28380bdc50 con 0x7f2840106f50 2024-09-16T00:39:35.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:35.953+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f284010a1a0 con 0x7f2840106f50 2024-09-16T00:39:36.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.136+0000 7f283d7fa640 1 -- 172.21.15.31:0/4150679331 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f28380bd550 con 0x7f2840106f50 2024-09-16T00:39:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28280a32c0 msgr2=0x7f28280a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28280a32c0 0x7f28280a5780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f283000baa0 tx=0x7f2830005d90 comp rx=0 tx=0).stop 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 msgr2=0x7f284019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 0x7f284019cd50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2838009ca0 tx=0x7f28380039f0 comp rx=0 tx=0).stop 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 shutdown_connections 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2840106f50 0x7f284019cd50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2840100ca0 0x7f28401989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28280a32c0 0x7f28280a5780 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 --2- 172.21.15.31:0/4150679331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28401002f0 0x7f2840198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 >> 172.21.15.31:0/4150679331 conn(0x7f2840077210 msgr2=0x7f2840105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 shutdown_connections 2024-09-16T00:39:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:36.138+0000 7f28462d8640 1 -- 172.21.15.31:0/4150679331 wait complete. 2024-09-16T00:39:36.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:36 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4150679331' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:39:36.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:36 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4150679331' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:39:36.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:36 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4150679331' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:39:36.971 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-16T00:39:36.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-16T00:39:37.233 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:37.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:37 smithi031 ceph-mon[30956]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 85 B/s wr, 0 op/s 2024-09-16T00:39:37.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:37 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4150679331' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:39:37.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:37 smithi031 ceph-mon[30956]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-16T00:39:37.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:37 smithi081 ceph-mon[35539]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 85 B/s wr, 0 op/s 2024-09-16T00:39:37.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:37 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4150679331' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:39:37.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:37 smithi081 ceph-mon[35539]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-16T00:39:37.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:37 smithi102 ceph-mon[35364]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 85 B/s wr, 0 op/s 2024-09-16T00:39:37.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:37 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4150679331' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:39:37.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:37 smithi102 ceph-mon[35364]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-16T00:39:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.745+0000 7fc5cedb6640 1 -- 172.21.15.31:0/227705083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c81036c0 msgr2=0x7fc5c810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.745+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/227705083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c81036c0 0x7fc5c810bbb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009900 tx=0x7fc5b402a480 comp rx=0 tx=0).stop 2024-09-16T00:39:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.745+0000 7fc5cedb6640 1 -- 172.21.15.31:0/227705083 shutdown_connections 2024-09-16T00:39:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.745+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/227705083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c810c180 0x7fc5c810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.745+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/227705083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c81036c0 0x7fc5c810bbb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.745+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/227705083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102da0 0x7fc5c8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.745+0000 7fc5cedb6640 1 -- 172.21.15.31:0/227705083 >> 172.21.15.31:0/227705083 conn(0x7fc5c80fc7a0 msgr2=0x7fc5c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.745+0000 7fc5cedb6640 1 -- 172.21.15.31:0/227705083 shutdown_connections 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.745+0000 7fc5cedb6640 1 -- 172.21.15.31:0/227705083 wait complete. 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cedb6640 1 Processor -- start 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cedb6640 1 -- start start 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cedb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102da0 0x7fc5c81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cedb6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c81036c0 0x7fc5c81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cedb6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c180 0x7fc5c819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cedb6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8111fa0 con 0x7fc5c810c180 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cedb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8111e20 con 0x7fc5c8102da0 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cedb6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8112120 con 0x7fc5c81036c0 2024-09-16T00:39:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cd353640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c180 0x7fc5c819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cd353640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c180 0x7fc5c819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54760/0 (socket says 172.21.15.31:54760) 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5cd353640 1 -- 172.21.15.31:0/202832286 learned_addr learned my addr 172.21.15.31:0/202832286 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5c7fff640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c81036c0 0x7fc5c81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.746+0000 7fc5ccb52640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102da0 0x7fc5c81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5cd353640 1 -- 172.21.15.31:0/202832286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c81036c0 msgr2=0x7fc5c81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5cd353640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c81036c0 0x7fc5c81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5cd353640 1 -- 172.21.15.31:0/202832286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102da0 msgr2=0x7fc5c81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5cd353640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102da0 0x7fc5c81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5cd353640 1 -- 172.21.15.31:0/202832286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c819d460 con 0x7fc5c810c180 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5c7fff640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c81036c0 0x7fc5c81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5cd353640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c180 0x7fc5c819cd80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000b970 tx=0x7fc5c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5c5ffb640 1 -- 172.21.15.31:0/202832286 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0019070 con 0x7fc5c810c180 2024-09-16T00:39:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5c5ffb640 1 -- 172.21.15.31:0/202832286 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c000cb40 con 0x7fc5c810c180 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c819d750 con 0x7fc5c810c180 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5c5ffb640 1 -- 172.21.15.31:0/202832286 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0007650 con 0x7fc5c810c180 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.747+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c81a5030 con 0x7fc5c810c180 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.748+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c8103180 con 0x7fc5c810c180 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.749+0000 7fc5c5ffb640 1 -- 172.21.15.31:0/202832286 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc5c001c070 con 0x7fc5c810c180 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.750+0000 7fc5c5ffb640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b80a3240 0x7fc5b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.750+0000 7fc5ccb52640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b80a3240 0x7fc5b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.750+0000 7fc5c5ffb640 1 -- 172.21.15.31:0/202832286 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fc5c00cf270 con 0x7fc5c810c180 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.750+0000 7fc5ccb52640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b80a3240 0x7fc5b80a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009b60 tx=0x7fc5bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.754+0000 7fc5c5ffb640 1 -- 172.21.15.31:0/202832286 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5c009bcf0 con 0x7fc5c810c180 2024-09-16T00:39:37.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.955+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc5c8062630 con 0x7fc5b80a3240 2024-09-16T00:39:37.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.975+0000 7fc5c5ffb640 1 -- 172.21.15.31:0/202832286 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc5c8062630 con 0x7fc5b80a3240 2024-09-16T00:39:37.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.977+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b80a3240 msgr2=0x7fc5b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.977+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b80a3240 0x7fc5b80a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009b60 tx=0x7fc5bc005f90 comp rx=0 tx=0).stop 2024-09-16T00:39:37.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.977+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c180 msgr2=0x7fc5c819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:37.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.977+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c180 0x7fc5c819cd80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000b970 tx=0x7fc5c000be30 comp rx=0 tx=0).stop 2024-09-16T00:39:37.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.977+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 shutdown_connections 2024-09-16T00:39:37.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.977+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c180 0x7fc5c819cd80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.977+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b80a3240 0x7fc5b80a5700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.977+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c81036c0 0x7fc5c81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.978+0000 7fc5cedb6640 1 --2- 172.21.15.31:0/202832286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102da0 0x7fc5c81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:37.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.978+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 >> 172.21.15.31:0/202832286 conn(0x7fc5c80fc7a0 msgr2=0x7fc5c80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:37.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.978+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 shutdown_connections 2024-09-16T00:39:37.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:37.978+0000 7fc5cedb6640 1 -- 172.21.15.31:0/202832286 wait complete. 2024-09-16T00:39:38.363 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:39:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:39:37.961+0000 7faf53686640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:39:38.364 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:39:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:39:37.961+0000 7faf53686640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:39:38.364 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:39:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:39:37.961+0000 7faf53686640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:39:38.364 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:39:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:39:37.961+0000 7faf53686640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:39:38.364 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:39:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x[31187]: 2024-09-16T00:39:37.961+0000 7faf53686640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:39:38.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-16T00:39:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:39.368 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:39 smithi031 ceph-mon[30956]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 op/s 2024-09-16T00:39:39.368 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:39 smithi031 ceph-mon[30956]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:39.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 -- 172.21.15.31:0/4195838094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db81055b0 msgr2=0x7f0db810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:39.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/4195838094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db81055b0 0x7f0db810d130 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009ca0 tx=0x7f0db002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 -- 172.21.15.31:0/4195838094 shutdown_connections 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/4195838094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db81055b0 0x7f0db810d130 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/4195838094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8100e30 0x7f0db8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/4195838094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db8100480 0x7f0db8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 -- 172.21.15.31:0/4195838094 >> 172.21.15.31:0/4195838094 conn(0x7f0db8077210 msgr2=0x7f0db80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 -- 172.21.15.31:0/4195838094 shutdown_connections 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 -- 172.21.15.31:0/4195838094 wait complete. 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 Processor -- start 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.455+0000 7f0dbfb99640 1 -- start start 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbfb99640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8100480 0x7f0db80fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbfb99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db8100e30 0x7f0db80fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbfb99640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db81055b0 0x7f0db80fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:39.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbfb99640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8110e90 con 0x7f0db8100e30 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbfb99640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8110d10 con 0x7f0db81055b0 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbfb99640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8111010 con 0x7f0db8100480 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd134640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db8100e30 0x7f0db80fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd134640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db8100e30 0x7f0db80fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54786/0 (socket says 172.21.15.31:54786) 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd134640 1 -- 172.21.15.31:0/3925171149 learned_addr learned my addr 172.21.15.31:0/3925171149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd935640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8100480 0x7f0db80fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbe136640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db81055b0 0x7f0db80fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd134640 1 -- 172.21.15.31:0/3925171149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8100480 msgr2=0x7f0db80fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd134640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8100480 0x7f0db80fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd134640 1 -- 172.21.15.31:0/3925171149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db81055b0 msgr2=0x7f0db80fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd134640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db81055b0 0x7f0db80fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd134640 1 -- 172.21.15.31:0/3925171149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db81a6930 con 0x7f0db8100e30 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbe136640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db81055b0 0x7f0db80fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0dbd134640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db8100e30 0x7f0db80fcbd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0da800d950 tx=0x7f0da800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.456+0000 7f0daeffd640 1 -- 172.21.15.31:0/3925171149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8017030 con 0x7f0db8100e30 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.457+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db81a6c20 con 0x7f0db8100e30 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.457+0000 7f0daeffd640 1 -- 172.21.15.31:0/3925171149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da8004020 con 0x7f0db8100e30 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.457+0000 7f0daeffd640 1 -- 172.21.15.31:0/3925171149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da80047a0 con 0x7f0db8100e30 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.457+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db81a71b0 con 0x7f0db8100e30 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.458+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db8100860 con 0x7f0db8100e30 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.459+0000 7f0daeffd640 1 -- 172.21.15.31:0/3925171149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0da80041d0 con 0x7f0db8100e30 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.459+0000 7f0daeffd640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0da40a3240 0x7f0da40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.460+0000 7f0dbd935640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0da40a3240 0x7f0da40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:39.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.460+0000 7f0daeffd640 1 -- 172.21.15.31:0/3925171149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f0da80cfb10 con 0x7f0db8100e30 2024-09-16T00:39:39.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.460+0000 7f0dbd935640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0da40a3240 0x7f0da40a5700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0db4005d00 tx=0x7f0db400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:39.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.463+0000 7f0daeffd640 1 -- 172.21.15.31:0/3925171149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0da809c5f0 con 0x7f0db8100e30 2024-09-16T00:39:39.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:39 smithi081 ceph-mon[35539]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 op/s 2024-09-16T00:39:39.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:39 smithi081 ceph-mon[35539]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:39.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:39 smithi102 ceph-mon[35364]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 op/s 2024-09-16T00:39:39.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:39 smithi102 ceph-mon[35364]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:39.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.666+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0db810a870 con 0x7f0da40a3240 2024-09-16T00:39:39.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.690+0000 7f0daeffd640 1 -- 172.21.15.31:0/3925171149 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0db400bc40 con 0x7f0da40a3240 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0da40a3240 msgr2=0x7f0da40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0da40a3240 0x7f0da40a5700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0db4005d00 tx=0x7f0db400b410 comp rx=0 tx=0).stop 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db8100e30 msgr2=0x7f0db80fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db8100e30 0x7f0db80fcbd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0da800d950 tx=0x7f0da800de10 comp rx=0 tx=0).stop 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 shutdown_connections 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db81055b0 0x7f0db80fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0db8100e30 0x7f0db80fcbd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0da40a3240 0x7f0da40a5700 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 --2- 172.21.15.31:0/3925171149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8100480 0x7f0db80fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 >> 172.21.15.31:0/3925171149 conn(0x7f0db8077210 msgr2=0x7f0db8103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 shutdown_connections 2024-09-16T00:39:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:39.692+0000 7f0dbfb99640 1 -- 172.21.15.31:0/3925171149 wait complete. 2024-09-16T00:39:40.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-16T00:39:40.686 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.176+0000 7f4003b3f640 1 -- 172.21.15.31:0/3599285341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 msgr2=0x7f3ffc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.176+0000 7f4003b3f640 1 --2- 172.21.15.31:0/3599285341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 0x7f3ffc1067e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009900 tx=0x7f3ff002a410 comp rx=0 tx=0).stop 2024-09-16T00:39:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.176+0000 7f4003b3f640 1 -- 172.21.15.31:0/3599285341 shutdown_connections 2024-09-16T00:39:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.176+0000 7f4003b3f640 1 --2- 172.21.15.31:0/3599285341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ffc106f10 0x7f3ffc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.176+0000 7f4003b3f640 1 --2- 172.21.15.31:0/3599285341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 0x7f3ffc1067e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.176+0000 7f4003b3f640 1 --2- 172.21.15.31:0/3599285341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ffc101de0 0x7f3ffc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.176+0000 7f4003b3f640 1 -- 172.21.15.31:0/3599285341 >> 172.21.15.31:0/3599285341 conn(0x7f3ffc0fd8c0 msgr2=0x7f3ffc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.176+0000 7f4003b3f640 1 -- 172.21.15.31:0/3599285341 shutdown_connections 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.176+0000 7f4003b3f640 1 -- 172.21.15.31:0/3599285341 wait complete. 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f4003b3f640 1 Processor -- start 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f4003b3f640 1 -- start start 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f4003b3f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ffc101de0 0x7f3ffc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f4003b3f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 0x7f3ffc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f4003b3f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ffc106f10 0x7f3ffc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f4003b3f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc111f90 con 0x7f3ffc102790 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f4003b3f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc111e10 con 0x7f3ffc106f10 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f4003b3f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc112110 con 0x7f3ffc101de0 2024-09-16T00:39:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40010da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 0x7f3ffc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40010da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 0x7f3ffc1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54806/0 (socket says 172.21.15.31:54806) 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40010da640 1 -- 172.21.15.31:0/910716044 learned_addr learned my addr 172.21.15.31:0/910716044 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40018db640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ffc101de0 0x7f3ffc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40010da640 1 -- 172.21.15.31:0/910716044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ffc101de0 msgr2=0x7f3ffc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40010da640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ffc101de0 0x7f3ffc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40010da640 1 -- 172.21.15.31:0/910716044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ffc106f10 msgr2=0x7f3ffc19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40020dc640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ffc106f10 0x7f3ffc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40010da640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ffc106f10 0x7f3ffc19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40010da640 1 -- 172.21.15.31:0/910716044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc19d420 con 0x7f3ffc102790 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.177+0000 7f40010da640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 0x7f3ffc1989b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0006010 tx=0x7f3ff00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.178+0000 7f3feeffd640 1 -- 172.21.15.31:0/910716044 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff003f030 con 0x7f3ffc102790 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.178+0000 7f3feeffd640 1 -- 172.21.15.31:0/910716044 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff0003ce0 con 0x7f3ffc102790 2024-09-16T00:39:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.178+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc19d6b0 con 0x7f3ffc102790 2024-09-16T00:39:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.178+0000 7f3feeffd640 1 -- 172.21.15.31:0/910716044 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0035070 con 0x7f3ffc102790 2024-09-16T00:39:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.178+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1a4ff0 con 0x7f3ffc102790 2024-09-16T00:39:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.179+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc1021c0 con 0x7f3ffc102790 2024-09-16T00:39:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.180+0000 7f3feeffd640 1 -- 172.21.15.31:0/910716044 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3ff0042030 con 0x7f3ffc102790 2024-09-16T00:39:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.180+0000 7f3feeffd640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3fe80a32d0 0x7f3fe80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.181+0000 7f3feeffd640 1 -- 172.21.15.31:0/910716044 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f3ff00f03a0 con 0x7f3ffc102790 2024-09-16T00:39:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.181+0000 7f40018db640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3fe80a32d0 0x7f3fe80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.181+0000 7f40018db640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3fe80a32d0 0x7f3fe80a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800bbe0 tx=0x7f3ff800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.185+0000 7f3feeffd640 1 -- 172.21.15.31:0/910716044 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ff00bcde0 con 0x7f3ffc102790 2024-09-16T00:39:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:41 smithi031 ceph-mon[30956]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-16T00:39:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:41 smithi031 ceph-mon[30956]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:41 smithi031 ceph-mon[30956]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:39:41.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.387+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3ffc062630 con 0x7f3fe80a32d0 2024-09-16T00:39:41.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.391+0000 7f3feeffd640 1 -- 172.21.15.31:0/910716044 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3ffc062630 con 0x7f3fe80a32d0 2024-09-16T00:39:41.393 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_0/886c7186-13e9-4c12-a4aa-4f004314bada 2024-09-16T00:39:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.393+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3fe80a32d0 msgr2=0x7f3fe80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.393+0000 7f4003b3f640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3fe80a32d0 0x7f3fe80a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800bbe0 tx=0x7f3ff800b390 comp rx=0 tx=0).stop 2024-09-16T00:39:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.393+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 msgr2=0x7f3ffc1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.393+0000 7f4003b3f640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 0x7f3ffc1989b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0006010 tx=0x7f3ff00042f0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.394+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 shutdown_connections 2024-09-16T00:39:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.394+0000 7f4003b3f640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ffc106f10 0x7f3ffc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.394+0000 7f4003b3f640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc102790 0x7f3ffc1989b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.394+0000 7f4003b3f640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3fe80a32d0 0x7f3fe80a5790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.394+0000 7f4003b3f640 1 --2- 172.21.15.31:0/910716044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ffc101de0 0x7f3ffc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.394+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 >> 172.21.15.31:0/910716044 conn(0x7f3ffc0fd8c0 msgr2=0x7f3ffc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.394+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 shutdown_connections 2024-09-16T00:39:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:41.394+0000 7f4003b3f640 1 -- 172.21.15.31:0/910716044 wait complete. 2024-09-16T00:39:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:41 smithi081 ceph-mon[35539]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-16T00:39:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:41 smithi081 ceph-mon[35539]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:41 smithi081 ceph-mon[35539]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:39:41.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:41 smithi102 ceph-mon[35364]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-16T00:39:41.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:41 smithi102 ceph-mon[35364]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:41.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:41 smithi102 ceph-mon[35364]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:39:42.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-16T00:39:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:42.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:42 smithi031 ceph-mon[30956]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:42.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:42 smithi081 ceph-mon[35539]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:42.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:42 smithi102 ceph-mon[35364]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:42.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 -- 172.21.15.31:0/2234125884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 msgr2=0x7f00a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 --2- 172.21.15.31:0/2234125884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 0x7f00a810da40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009ca0 tx=0x7f00a002a950 comp rx=0 tx=0).stop 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 -- 172.21.15.31:0/2234125884 shutdown_connections 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 --2- 172.21.15.31:0/2234125884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 0x7f00a810da40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 --2- 172.21.15.31:0/2234125884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00a8102570 0x7f00a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 --2- 172.21.15.31:0/2234125884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00a8100e20 0x7f00a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 -- 172.21.15.31:0/2234125884 >> 172.21.15.31:0/2234125884 conn(0x7f00a80fc7d0 msgr2=0x7f00a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 -- 172.21.15.31:0/2234125884 shutdown_connections 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 -- 172.21.15.31:0/2234125884 wait complete. 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 Processor -- start 2024-09-16T00:39:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.769+0000 7f00aebca640 1 -- start start 2024-09-16T00:39:42.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00aebca640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00a8100e20 0x7f00a81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00aebca640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00a8102570 0x7f00a81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00aebca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 0x7f00a819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00aebca640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8110f80 con 0x7f00a8102ef0 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00aebca640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8110e00 con 0x7f00a8100e20 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00aebca640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8111100 con 0x7f00a8102570 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ad167640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 0x7f00a819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ad167640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 0x7f00a819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54824/0 (socket says 172.21.15.31:54824) 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ad167640 1 -- 172.21.15.31:0/2345964587 learned_addr learned my addr 172.21.15.31:0/2345964587 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00a7fff640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00a8102570 0x7f00a81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ac966640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00a8100e20 0x7f00a81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ad167640 1 -- 172.21.15.31:0/2345964587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00a8102570 msgr2=0x7f00a81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ad167640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00a8102570 0x7f00a81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ad167640 1 -- 172.21.15.31:0/2345964587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00a8100e20 msgr2=0x7f00a81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ad167640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00a8100e20 0x7f00a81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ad167640 1 -- 172.21.15.31:0/2345964587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a819d460 con 0x7f00a8102ef0 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.770+0000 7f00ad167640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 0x7f00a819cd80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f00a002ae70 tx=0x7f00a002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.771+0000 7f00a5ffb640 1 -- 172.21.15.31:0/2345964587 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0034040 con 0x7f00a8102ef0 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.771+0000 7f00a5ffb640 1 -- 172.21.15.31:0/2345964587 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a0030920 con 0x7f00a8102ef0 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.771+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a819d6f0 con 0x7f00a8102ef0 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.771+0000 7f00a5ffb640 1 -- 172.21.15.31:0/2345964587 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0045480 con 0x7f00a8102ef0 2024-09-16T00:39:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.771+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a81a5030 con 0x7f00a8102ef0 2024-09-16T00:39:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.772+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a8101200 con 0x7f00a8102ef0 2024-09-16T00:39:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.773+0000 7f00a5ffb640 1 -- 172.21.15.31:0/2345964587 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f00a0037020 con 0x7f00a8102ef0 2024-09-16T00:39:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.774+0000 7f00a5ffb640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f00980a3350 0x7f00980a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.774+0000 7f00a5ffb640 1 -- 172.21.15.31:0/2345964587 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f00a00f0970 con 0x7f00a8102ef0 2024-09-16T00:39:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.774+0000 7f00ac966640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f00980a3350 0x7f00980a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.774+0000 7f00ac966640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f00980a3350 0x7f00980a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f009c0058c0 tx=0x7f009c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:42.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.778+0000 7f00a5ffb640 1 -- 172.21.15.31:0/2345964587 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00a00bd3a0 con 0x7f00a8102ef0 2024-09-16T00:39:42.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:42.978+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f00a8062630 con 0x7f00980a3350 2024-09-16T00:39:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.003+0000 7f00a5ffb640 1 -- 172.21.15.31:0/2345964587 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f00a8062630 con 0x7f00980a3350 2024-09-16T00:39:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f00980a3350 msgr2=0x7f00980a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f00980a3350 0x7f00980a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f009c0058c0 tx=0x7f009c00a2d0 comp rx=0 tx=0).stop 2024-09-16T00:39:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 msgr2=0x7f00a819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 0x7f00a819cd80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f00a002ae70 tx=0x7f00a002f5d0 comp rx=0 tx=0).stop 2024-09-16T00:39:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 shutdown_connections 2024-09-16T00:39:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00a8102ef0 0x7f00a819cd80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f00980a3350 0x7f00980a5810 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00a8102570 0x7f00a81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 --2- 172.21.15.31:0/2345964587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00a8100e20 0x7f00a81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.005+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 >> 172.21.15.31:0/2345964587 conn(0x7f00a80fc7d0 msgr2=0x7f00a806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.006+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 shutdown_connections 2024-09-16T00:39:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:43.006+0000 7f00aebca640 1 -- 172.21.15.31:0/2345964587 wait complete. 2024-09-16T00:39:43.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:43 smithi031 ceph-mon[30956]: pgmap v309: 129 pgs: 129 active+clean; 601 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-09-16T00:39:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:43 smithi081 ceph-mon[35539]: pgmap v309: 129 pgs: 129 active+clean; 601 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-09-16T00:39:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:43 smithi102 ceph-mon[35364]: pgmap v309: 129 pgs: 129 active+clean; 601 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-09-16T00:39:43.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-16T00:39:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:44.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:44 smithi031 ceph-mon[30956]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:44.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 -- 172.21.15.31:0/3561457782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c8103660 msgr2=0x7fc5c810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/3561457782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c8103660 0x7fc5c810bb50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009900 tx=0x7fc5bc02a480 comp rx=0 tx=0).stop 2024-09-16T00:39:44.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 -- 172.21.15.31:0/3561457782 shutdown_connections 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/3561457782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c810c170 0x7fc5c810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/3561457782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c8103660 0x7fc5c810bb50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/3561457782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102d40 0x7fc5c8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 -- 172.21.15.31:0/3561457782 >> 172.21.15.31:0/3561457782 conn(0x7fc5c80fc7a0 msgr2=0x7fc5c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 -- 172.21.15.31:0/3561457782 shutdown_connections 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 -- 172.21.15.31:0/3561457782 wait complete. 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.462+0000 7fc5ccf15640 1 Processor -- start 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5ccf15640 1 -- start start 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5ccf15640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102d40 0x7fc5c81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5ccf15640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c8103660 0x7fc5c8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5ccf15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c170 0x7fc5c8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5ccf15640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8111f70 con 0x7fc5c810c170 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5ccf15640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8111df0 con 0x7fc5c8102d40 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5ccf15640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c81120f0 con 0x7fc5c8103660 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c170 0x7fc5c8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c170 0x7fc5c8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54842/0 (socket says 172.21.15.31:54842) 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c6d9d640 1 -- 172.21.15.31:0/167310245 learned_addr learned my addr 172.21.15.31:0/167310245 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c659c640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102d40 0x7fc5c81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c5d9b640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c8103660 0x7fc5c8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c6d9d640 1 -- 172.21.15.31:0/167310245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c8103660 msgr2=0x7fc5c8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c6d9d640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c8103660 0x7fc5c8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c6d9d640 1 -- 172.21.15.31:0/167310245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102d40 msgr2=0x7fc5c81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c6d9d640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102d40 0x7fc5c81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c6d9d640 1 -- 172.21.15.31:0/167310245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c81a68a0 con 0x7fc5c810c170 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.463+0000 7fc5c6d9d640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c170 0x7fc5c8101420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000cce0 tx=0x7fc5c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.464+0000 7fc5b77fe640 1 -- 172.21.15.31:0/167310245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c000f450 con 0x7fc5c810c170 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.464+0000 7fc5b77fe640 1 -- 172.21.15.31:0/167310245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c000fad0 con 0x7fc5c810c170 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.464+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c81a6b90 con 0x7fc5c810c170 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.464+0000 7fc5b77fe640 1 -- 172.21.15.31:0/167310245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0018c10 con 0x7fc5c810c170 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.464+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c81a70d0 con 0x7fc5c810c170 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.465+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c8103120 con 0x7fc5c810c170 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.466+0000 7fc5b77fe640 1 -- 172.21.15.31:0/167310245 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc5c000f5f0 con 0x7fc5c810c170 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.467+0000 7fc5b77fe640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b00a3350 0x7fc5b00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.467+0000 7fc5c659c640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b00a3350 0x7fc5b00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.467+0000 7fc5b77fe640 1 -- 172.21.15.31:0/167310245 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fc5c00d02f0 con 0x7fc5c810c170 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.467+0000 7fc5c659c640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b00a3350 0x7fc5b00a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800bbe0 tx=0x7fc5b800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.471+0000 7fc5b77fe640 1 -- 172.21.15.31:0/167310245 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5c0014030 con 0x7fc5c810c170 2024-09-16T00:39:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:44 smithi081 ceph-mon[35539]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:44.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:44 smithi102 ceph-mon[35364]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:44.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.674+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc5c8107250 con 0x7fc5b00a3350 2024-09-16T00:39:44.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.678+0000 7fc5b77fe640 1 -- 172.21.15.31:0/167310245 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc5b8010c30 con 0x7fc5b00a3350 2024-09-16T00:39:44.678 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_1/ff2f5246-15d0-407a-b34e-c330a0c83c3c 2024-09-16T00:39:44.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b00a3350 msgr2=0x7fc5b00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b00a3350 0x7fc5b00a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800bbe0 tx=0x7fc5b800b390 comp rx=0 tx=0).stop 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c170 msgr2=0x7fc5c8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c170 0x7fc5c8101420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000cce0 tx=0x7fc5c000a620 comp rx=0 tx=0).stop 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 shutdown_connections 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5c810c170 0x7fc5c8101420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc5b00a3350 0x7fc5b00a5810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5c8103660 0x7fc5c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 --2- 172.21.15.31:0/167310245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5c8102d40 0x7fc5c81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 >> 172.21.15.31:0/167310245 conn(0x7fc5c80fc7a0 msgr2=0x7fc5c80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 shutdown_connections 2024-09-16T00:39:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:44.680+0000 7fc5ccf15640 1 -- 172.21.15.31:0/167310245 wait complete. 2024-09-16T00:39:45.353 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:39:45.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:45 smithi081 ceph-mon[35539]: pgmap v310: 129 pgs: 129 active+clean; 601 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T00:39:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:45 smithi102 ceph-mon[35364]: pgmap v310: 129 pgs: 129 active+clean; 601 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T00:39:45.611 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:45.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:45 smithi031 ceph-mon[30956]: pgmap v310: 129 pgs: 129 active+clean; 601 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-16T00:39:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.102+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3068776344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4102790 msgr2=0x7f2ba41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.102+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3068776344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4102790 0x7f2ba41067e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009900 tx=0x7f2b9802a410 comp rx=0 tx=0).stop 2024-09-16T00:39:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.102+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3068776344 shutdown_connections 2024-09-16T00:39:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.102+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3068776344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba4106f10 0x7f2ba410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.102+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3068776344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4102790 0x7f2ba41067e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.102+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3068776344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ba4101de0 0x7f2ba41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.102+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3068776344 >> 172.21.15.31:0/3068776344 conn(0x7f2ba40fd8c0 msgr2=0x7f2ba40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.102+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3068776344 shutdown_connections 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.102+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3068776344 wait complete. 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba9fb1640 1 Processor -- start 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba9fb1640 1 -- start start 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba9fb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4101de0 0x7f2ba4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba9fb1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ba4102790 0x7f2ba4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba9fb1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba4106f10 0x7f2ba419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba9fb1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba4111f90 con 0x7f2ba4101de0 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba9fb1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba4111e10 con 0x7f2ba4102790 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba9fb1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba4112110 con 0x7f2ba4106f10 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4101de0 0x7f2ba4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4101de0 0x7f2ba4198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54868/0 (socket says 172.21.15.31:54868) 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba37fe640 1 -- 172.21.15.31:0/3524891429 learned_addr learned my addr 172.21.15.31:0/3524891429 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba2ffd640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ba4102790 0x7f2ba4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.103+0000 7f2ba3fff640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba4106f10 0x7f2ba419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba37fe640 1 -- 172.21.15.31:0/3524891429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba4106f10 msgr2=0x7f2ba419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba37fe640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba4106f10 0x7f2ba419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba37fe640 1 -- 172.21.15.31:0/3524891429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ba4102790 msgr2=0x7f2ba4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba37fe640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ba4102790 0x7f2ba4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba37fe640 1 -- 172.21.15.31:0/3524891429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba419d3d0 con 0x7f2ba4101de0 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba3fff640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba4106f10 0x7f2ba419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba2ffd640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ba4102790 0x7f2ba4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba37fe640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4101de0 0x7f2ba4198420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c920 tx=0x7f2b9400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba0ff9640 1 -- 172.21.15.31:0/3524891429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94007c70 con 0x7f2ba4101de0 2024-09-16T00:39:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba0ff9640 1 -- 172.21.15.31:0/3524891429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b94007e10 con 0x7f2ba4101de0 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba419d6c0 con 0x7f2ba4101de0 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba0ff9640 1 -- 172.21.15.31:0/3524891429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94018890 con 0x7f2ba4101de0 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.104+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba41a4fa0 con 0x7f2ba4101de0 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.105+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba41021c0 con 0x7f2ba4101de0 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.106+0000 7f2ba0ff9640 1 -- 172.21.15.31:0/3524891429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2b94018a70 con 0x7f2ba4101de0 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.107+0000 7f2ba0ff9640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b8c0a3350 0x7f2b8c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.107+0000 7f2ba2ffd640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b8c0a3350 0x7f2b8c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.107+0000 7f2ba0ff9640 1 -- 172.21.15.31:0/3524891429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f2b940cfeb0 con 0x7f2ba4101de0 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.107+0000 7f2ba2ffd640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b8c0a3350 0x7f2b8c0a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40fe0d0 tx=0x7f2b98005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.111+0000 7f2ba0ff9640 1 -- 172.21.15.31:0/3524891429 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b94014030 con 0x7f2ba4101de0 2024-09-16T00:39:46.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.323+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ba410a150 con 0x7f2ba4101de0 2024-09-16T00:39:46.323 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:46 smithi031 ceph-mon[30956]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:46.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.323+0000 7f2ba0ff9640 1 -- 172.21.15.31:0/3524891429 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f2b9409c830 con 0x7f2ba4101de0 2024-09-16T00:39:46.324 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:46.324 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:39:46.325 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:39:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b8c0a3350 msgr2=0x7f2b8c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b8c0a3350 0x7f2b8c0a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40fe0d0 tx=0x7f2b98005a20 comp rx=0 tx=0).stop 2024-09-16T00:39:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4101de0 msgr2=0x7f2ba4198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4101de0 0x7f2ba4198420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c920 tx=0x7f2b9400cde0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 shutdown_connections 2024-09-16T00:39:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2b8c0a3350 0x7f2b8c0a5810 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ba4106f10 0x7f2ba419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ba4102790 0x7f2ba4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 --2- 172.21.15.31:0/3524891429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ba4101de0 0x7f2ba4198420 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.325+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 >> 172.21.15.31:0/3524891429 conn(0x7f2ba40fd8c0 msgr2=0x7f2ba40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:46.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.326+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 shutdown_connections 2024-09-16T00:39:46.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:46.326+0000 7f2ba9fb1640 1 -- 172.21.15.31:0/3524891429 wait complete. 2024-09-16T00:39:46.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:46 smithi081 ceph-mon[35539]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:46.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:46 smithi102 ceph-mon[35364]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:47.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:39:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:47.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:47 smithi031 ceph-mon[30956]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-16T00:39:47.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:47 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3524891429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:47.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:47 smithi081 ceph-mon[35539]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-16T00:39:47.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:47 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3524891429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:47.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:47 smithi102 ceph-mon[35364]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-16T00:39:47.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:47 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3524891429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:39:47.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.763+0000 7fc73c21e640 1 -- 172.21.15.31:0/3425465070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7341008d0 msgr2=0x7fc734100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.763+0000 7fc73c21e640 1 --2- 172.21.15.31:0/3425465070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7341008d0 0x7fc734100cb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc730009900 tx=0x7fc73002a410 comp rx=0 tx=0).stop 2024-09-16T00:39:47.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 -- 172.21.15.31:0/3425465070 shutdown_connections 2024-09-16T00:39:47.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 --2- 172.21.15.31:0/3425465070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc73410e480 0x7fc734110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 --2- 172.21.15.31:0/3425465070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734101280 0x7fc73410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 --2- 172.21.15.31:0/3425465070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7341008d0 0x7fc734100cb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 -- 172.21.15.31:0/3425465070 >> 172.21.15.31:0/3425465070 conn(0x7fc7340fc7a0 msgr2=0x7fc7340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 -- 172.21.15.31:0/3425465070 shutdown_connections 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 -- 172.21.15.31:0/3425465070 wait complete. 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 Processor -- start 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 -- start start 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.764+0000 7fc73c21e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7341008d0 0x7fc73419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc73c21e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc734101280 0x7fc73419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc73c21e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc73410e480 0x7fc73419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc73c21e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341141b0 con 0x7fc734101280 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc73c21e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc734114030 con 0x7fc73410e480 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc73c21e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc734114330 con 0x7fc7341008d0 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc7397b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc734101280 0x7fc73419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc7397b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc734101280 0x7fc73419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54144/0 (socket says 172.21.15.31:54144) 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc7397b9640 1 -- 172.21.15.31:0/2902431780 learned_addr learned my addr 172.21.15.31:0/2902431780 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc739fba640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7341008d0 0x7fc73419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc73a7bb640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc73410e480 0x7fc73419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc7397b9640 1 -- 172.21.15.31:0/2902431780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7341008d0 msgr2=0x7fc73419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc7397b9640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7341008d0 0x7fc73419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc7397b9640 1 -- 172.21.15.31:0/2902431780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc73410e480 msgr2=0x7fc73419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc7397b9640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc73410e480 0x7fc73419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc7397b9640 1 -- 172.21.15.31:0/2902431780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73419f640 con 0x7fc734101280 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc73a7bb640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc73410e480 0x7fc73419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc739fba640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7341008d0 0x7fc73419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc7397b9640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc734101280 0x7fc73419abd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d770 tx=0x7fc72400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc72affd640 1 -- 172.21.15.31:0/2902431780 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724017070 con 0x7fc734101280 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73419f930 con 0x7fc734101280 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc72affd640 1 -- 172.21.15.31:0/2902431780 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc724005500 con 0x7fc734101280 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc72affd640 1 -- 172.21.15.31:0/2902431780 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724004e00 con 0x7fc734101280 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.765+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a7210 con 0x7fc734101280 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.766+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc734100cb0 con 0x7fc734101280 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.768+0000 7fc72affd640 1 -- 172.21.15.31:0/2902431780 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc724004fa0 con 0x7fc734101280 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.768+0000 7fc72affd640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc7200a32d0 0x7fc7200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.769+0000 7fc739fba640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc7200a32d0 0x7fc7200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.769+0000 7fc72affd640 1 -- 172.21.15.31:0/2902431780 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fc7240cfca0 con 0x7fc734101280 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.769+0000 7fc739fba640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc7200a32d0 0x7fc7200a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc730005ab0 tx=0x7fc730005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.772+0000 7fc72affd640 1 -- 172.21.15.31:0/2902431780 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc72409c5c0 con 0x7fc734101280 2024-09-16T00:39:47.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.951+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc73419b9f0 con 0x7fc734101280 2024-09-16T00:39:47.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.953+0000 7fc72affd640 1 -- 172.21.15.31:0/2902431780 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24150 (secure 0 0 0) 0x7fc73419b9f0 con 0x7fc734101280 2024-09-16T00:39:47.954 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:47.954 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:39:02.716519+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:39:47.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc7200a32d0 msgr2=0x7fc7200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc7200a32d0 0x7fc7200a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc730005ab0 tx=0x7fc730005a20 comp rx=0 tx=0).stop 2024-09-16T00:39:47.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc734101280 msgr2=0x7fc73419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc734101280 0x7fc73419abd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d770 tx=0x7fc72400dc30 comp rx=0 tx=0).stop 2024-09-16T00:39:47.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 shutdown_connections 2024-09-16T00:39:47.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc73410e480 0x7fc73419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc734101280 0x7fc73419abd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc7200a32d0 0x7fc7200a5790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 --2- 172.21.15.31:0/2902431780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7341008d0 0x7fc73419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 >> 172.21.15.31:0/2902431780 conn(0x7fc7340fc7a0 msgr2=0x7fc7340feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:47.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 shutdown_connections 2024-09-16T00:39:47.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:47.956+0000 7fc73c21e640 1 -- 172.21.15.31:0/2902431780 wait complete. 2024-09-16T00:39:48.277 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:48 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2902431780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:48.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:48 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2902431780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:48.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:48 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2902431780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:48.629 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:39:48.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:39:48.901 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:49.301 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:49 smithi031 ceph-mon[30956]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-16T00:39:49.301 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:49 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:39:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1486522122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 msgr2=0x7fdf4810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1486522122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 0x7fdf4810da40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009ca0 tx=0x7fdf4002a950 comp rx=0 tx=0).stop 2024-09-16T00:39:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1486522122 shutdown_connections 2024-09-16T00:39:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1486522122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 0x7fdf4810da40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1486522122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf48102570 0x7fdf481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1486522122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf48100e20 0x7fdf48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1486522122 >> 172.21.15.31:0/1486522122 conn(0x7fdf480fc7d0 msgr2=0x7fdf480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1486522122 shutdown_connections 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1486522122 wait complete. 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.382+0000 7fdf4d73e640 1 Processor -- start 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf4d73e640 1 -- start start 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf4d73e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf48100e20 0x7fdf48198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf4d73e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf48102570 0x7fdf48198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf4d73e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 0x7fdf4819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf4d73e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf48110fa0 con 0x7fdf48102ef0 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf4d73e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdf48110e20 con 0x7fdf48102570 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf4d73e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdf48111120 con 0x7fdf48100e20 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 0x7fdf4819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf467fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf48102570 0x7fdf48198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 0x7fdf4819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54170/0 (socket says 172.21.15.31:54170) 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf477fe640 1 -- 172.21.15.31:0/1405779814 learned_addr learned my addr 172.21.15.31:0/1405779814 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf46ffd640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf48100e20 0x7fdf48198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf467fc640 1 -- 172.21.15.31:0/1405779814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf48100e20 msgr2=0x7fdf48198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf467fc640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf48100e20 0x7fdf48198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf467fc640 1 -- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 msgr2=0x7fdf4819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf467fc640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 0x7fdf4819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf467fc640 1 -- 172.21.15.31:0/1405779814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4819d510 con 0x7fdf48102570 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.383+0000 7fdf477fe640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 0x7fdf4819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.384+0000 7fdf467fc640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf48102570 0x7fdf48198aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000d6d0 tx=0x7fdf3000dad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.384+0000 7fdf3ffff640 1 -- 172.21.15.31:0/1405779814 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30017070 con 0x7fdf48102570 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.384+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4819d800 con 0x7fdf48102570 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.384+0000 7fdf3ffff640 1 -- 172.21.15.31:0/1405779814 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf300047a0 con 0x7fdf48102570 2024-09-16T00:39:49.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.384+0000 7fdf3ffff640 1 -- 172.21.15.31:0/1405779814 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30009c80 con 0x7fdf48102570 2024-09-16T00:39:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.384+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf481a50e0 con 0x7fdf48102570 2024-09-16T00:39:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.385+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf48101200 con 0x7fdf48102570 2024-09-16T00:39:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.386+0000 7fdf3ffff640 1 -- 172.21.15.31:0/1405779814 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdf30004950 con 0x7fdf48102570 2024-09-16T00:39:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.386+0000 7fdf3ffff640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf340a32d0 0x7fdf340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.387+0000 7fdf46ffd640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf340a32d0 0x7fdf340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.387+0000 7fdf3ffff640 1 -- 172.21.15.31:0/1405779814 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fdf300cf570 con 0x7fdf48102570 2024-09-16T00:39:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.387+0000 7fdf46ffd640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf340a32d0 0x7fdf340a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009990 tx=0x7fdf38009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:49.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.391+0000 7fdf3ffff640 1 -- 172.21.15.31:0/1405779814 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf30009e20 con 0x7fdf48102570 2024-09-16T00:39:49.474 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:49 smithi102 ceph-mon[35364]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-16T00:39:49.474 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:49 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:39:49.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:49 smithi081 ceph-mon[35539]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-16T00:39:49.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:49 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:39:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.564+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf48062630 con 0x7fdf340a32d0 2024-09-16T00:39:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.573+0000 7fdf3ffff640 1 -- 172.21.15.31:0/1405779814 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340255 (secure 0 0 0) 0x7fdf48062630 con 0x7fdf340a32d0 2024-09-16T00:39:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:49.580 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:39:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf340a32d0 msgr2=0x7fdf340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf340a32d0 0x7fdf340a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009990 tx=0x7fdf38009110 comp rx=0 tx=0).stop 2024-09-16T00:39:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf48102570 msgr2=0x7fdf48198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf48102570 0x7fdf48198aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000d6d0 tx=0x7fdf3000dad0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 shutdown_connections 2024-09-16T00:39:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf48102ef0 0x7fdf4819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf48102570 0x7fdf48198aa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf340a32d0 0x7fdf340a5790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 --2- 172.21.15.31:0/1405779814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf48100e20 0x7fdf48198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 >> 172.21.15.31:0/1405779814 conn(0x7fdf480fc7d0 msgr2=0x7fdf4806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 shutdown_connections 2024-09-16T00:39:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:49.577+0000 7fdf4d73e640 1 -- 172.21.15.31:0/1405779814 wait complete. 2024-09-16T00:39:50.417 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:50 smithi031 ceph-mon[30956]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:39:50.417 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.417 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.417 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.417 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:50 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.446 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-16T00:39:47.664330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640498,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":161,"num_read_kb":177,"num_write":254,"num_write_kb":3014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400028,"kb_used_data":13328,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541796,"statfs":{"total":1151940427776,"available":1151530799104,"internally_reserved":0,"allocated":13647872,"data_stored":7672748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45488,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":59,"num_write":24,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003346"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.324102+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:39:03.324102+0000","last_peered":"2024-09-16T00:39:03.324102+0000","last_clean":"2024-09-16T00:39:03.324102+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:39:03.324102+0000","last_undegraded":"2024-09-16T00:39:03.324102+0000","last_fullsized":"2024-09-16T00:39:03.324102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037896+0000","last_change":"2024-09-16T00:38:52.968681+0000","last_active":"2024-09-16T00:39:04.037896+0000","last_peered":"2024-09-16T00:39:04.037896+0000","last_clean":"2024-09-16T00:39:04.037896+0000","last_became_active":"2024-09-16T00:38:52.968359+0000","last_became_peered":"2024-09-16T00:38:52.968359+0000","last_unstale":"2024-09-16T00:39:04.037896+0000","last_undegraded":"2024-09-16T00:39:04.037896+0000","last_fullsized":"2024-09-16T00:39:04.037896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:17.870176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950134+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:39:03.950134+0000","last_peered":"2024-09-16T00:39:03.950134+0000","last_clean":"2024-09-16T00:39:03.950134+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:39:03.950134+0000","last_undegraded":"2024-09-16T00:39:03.950134+0000","last_fullsized":"2024-09-16T00:39:03.950134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.038136+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:39:04.038136+0000","last_peered":"2024-09-16T00:39:04.038136+0000","last_clean":"2024-09-16T00:39:04.038136+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:39:04.038136+0000","last_undegraded":"2024-09-16T00:39:04.038136+0000","last_fullsized":"2024-09-16T00:39:04.038136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046126+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:39:03.046126+0000","last_peered":"2024-09-16T00:39:03.046126+0000","last_clean":"2024-09-16T00:39:03.046126+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:39:03.046126+0000","last_undegraded":"2024-09-16T00:39:03.046126+0000","last_fullsized":"2024-09-16T00:39:03.046126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877021+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:39:03.877021+0000","last_peered":"2024-09-16T00:39:03.877021+0000","last_clean":"2024-09-16T00:39:03.877021+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:39:03.877021+0000","last_undegraded":"2024-09-16T00:39:03.877021+0000","last_fullsized":"2024-09-16T00:39:03.877021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950908+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:39:03.950908+0000","last_peered":"2024-09-16T00:39:03.950908+0000","last_clean":"2024-09-16T00:39:03.950908+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:39:03.950908+0000","last_undegraded":"2024-09-16T00:39:03.950908+0000","last_fullsized":"2024-09-16T00:39:03.950908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576217+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:39:03.576217+0000","last_peered":"2024-09-16T00:39:03.576217+0000","last_clean":"2024-09-16T00:39:03.576217+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:39:03.576217+0000","last_undegraded":"2024-09-16T00:39:03.576217+0000","last_fullsized":"2024-09-16T00:39:03.576217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877200+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:39:03.877200+0000","last_peered":"2024-09-16T00:39:03.877200+0000","last_clean":"2024-09-16T00:39:03.877200+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:39:03.877200+0000","last_undegraded":"2024-09-16T00:39:03.877200+0000","last_fullsized":"2024-09-16T00:39:03.877200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951792+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:39:03.951792+0000","last_peered":"2024-09-16T00:39:03.951792+0000","last_clean":"2024-09-16T00:39:03.951792+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:39:03.951792+0000","last_undegraded":"2024-09-16T00:39:03.951792+0000","last_fullsized":"2024-09-16T00:39:03.951792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951852+0000","last_change":"2024-09-16T00:38:52.975634+0000","last_active":"2024-09-16T00:39:03.951852+0000","last_peered":"2024-09-16T00:39:03.951852+0000","last_clean":"2024-09-16T00:39:03.951852+0000","last_became_active":"2024-09-16T00:38:52.975314+0000","last_became_peered":"2024-09-16T00:38:52.975314+0000","last_unstale":"2024-09-16T00:39:03.951852+0000","last_undegraded":"2024-09-16T00:39:03.951852+0000","last_fullsized":"2024-09-16T00:39:03.951852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:28.716600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877164+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:39:03.877164+0000","last_peered":"2024-09-16T00:39:03.877164+0000","last_clean":"2024-09-16T00:39:03.877164+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:39:03.877164+0000","last_undegraded":"2024-09-16T00:39:03.877164+0000","last_fullsized":"2024-09-16T00:39:03.877164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046796+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:39:03.046796+0000","last_peered":"2024-09-16T00:39:03.046796+0000","last_clean":"2024-09-16T00:39:03.046796+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:39:03.046796+0000","last_undegraded":"2024-09-16T00:39:03.046796+0000","last_fullsized":"2024-09-16T00:39:03.046796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951025+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:39:03.951025+0000","last_peered":"2024-09-16T00:39:03.951025+0000","last_clean":"2024-09-16T00:39:03.951025+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:39:03.951025+0000","last_undegraded":"2024-09-16T00:39:03.951025+0000","last_fullsized":"2024-09-16T00:39:03.951025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877553+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:39:03.877553+0000","last_peered":"2024-09-16T00:39:03.877553+0000","last_clean":"2024-09-16T00:39:03.877553+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:39:03.877553+0000","last_undegraded":"2024-09-16T00:39:03.877553+0000","last_fullsized":"2024-09-16T00:39:03.877553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037967+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:39:04.037967+0000","last_peered":"2024-09-16T00:39:04.037967+0000","last_clean":"2024-09-16T00:39:04.037967+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:39:04.037967+0000","last_undegraded":"2024-09-16T00:39:04.037967+0000","last_fullsized":"2024-09-16T00:39:04.037967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950267+0000","last_change":"2024-09-16T00:38:50.874884+0000","last_active":"2024-09-16T00:39:03.950267+0000","last_peered":"2024-09-16T00:39:03.950267+0000","last_clean":"2024-09-16T00:39:03.950267+0000","last_became_active":"2024-09-16T00:38:50.874466+0000","last_became_peered":"2024-09-16T00:38:50.874466+0000","last_unstale":"2024-09-16T00:39:03.950267+0000","last_undegraded":"2024-09-16T00:39:03.950267+0000","last_fullsized":"2024-09-16T00:39:03.950267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:35.354581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877487+0000","last_change":"2024-09-16T00:39:00.047867+0000","last_active":"2024-09-16T00:39:03.877487+0000","last_peered":"2024-09-16T00:39:03.877487+0000","last_clean":"2024-09-16T00:39:03.877487+0000","last_became_active":"2024-09-16T00:39:00.047436+0000","last_became_peered":"2024-09-16T00:39:00.047436+0000","last_unstale":"2024-09-16T00:39:03.877487+0000","last_undegraded":"2024-09-16T00:39:03.877487+0000","last_fullsized":"2024-09-16T00:39:03.877487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:04.133052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046350+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:39:03.046350+0000","last_peered":"2024-09-16T00:39:03.046350+0000","last_clean":"2024-09-16T00:39:03.046350+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:39:03.046350+0000","last_undegraded":"2024-09-16T00:39:03.046350+0000","last_fullsized":"2024-09-16T00:39:03.046350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037850+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:39:04.037850+0000","last_peered":"2024-09-16T00:39:04.037850+0000","last_clean":"2024-09-16T00:39:04.037850+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:39:04.037850+0000","last_undegraded":"2024-09-16T00:39:04.037850+0000","last_fullsized":"2024-09-16T00:39:04.037850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323966+0000","last_change":"2024-09-16T00:39:00.053720+0000","last_active":"2024-09-16T00:39:03.323966+0000","last_peered":"2024-09-16T00:39:03.323966+0000","last_clean":"2024-09-16T00:39:03.323966+0000","last_became_active":"2024-09-16T00:39:00.053309+0000","last_became_peered":"2024-09-16T00:39:00.053309+0000","last_unstale":"2024-09-16T00:39:03.323966+0000","last_undegraded":"2024-09-16T00:39:03.323966+0000","last_fullsized":"2024-09-16T00:39:03.323966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:07.390804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951256+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:39:03.951256+0000","last_peered":"2024-09-16T00:39:03.951256+0000","last_clean":"2024-09-16T00:39:03.951256+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:39:03.951256+0000","last_undegraded":"2024-09-16T00:39:03.951256+0000","last_fullsized":"2024-09-16T00:39:03.951256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576050+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:39:03.576050+0000","last_peered":"2024-09-16T00:39:03.576050+0000","last_clean":"2024-09-16T00:39:03.576050+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:39:03.576050+0000","last_undegraded":"2024-09-16T00:39:03.576050+0000","last_fullsized":"2024-09-16T00:39:03.576050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877507+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:39:03.877507+0000","last_peered":"2024-09-16T00:39:03.877507+0000","last_clean":"2024-09-16T00:39:03.877507+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:39:03.877507+0000","last_undegraded":"2024-09-16T00:39:03.877507+0000","last_fullsized":"2024-09-16T00:39:03.877507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046022+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:39:03.046022+0000","last_peered":"2024-09-16T00:39:03.046022+0000","last_clean":"2024-09-16T00:39:03.046022+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:39:03.046022+0000","last_undegraded":"2024-09-16T00:39:03.046022+0000","last_fullsized":"2024-09-16T00:39:03.046022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877836+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:39:03.877836+0000","last_peered":"2024-09-16T00:39:03.877836+0000","last_clean":"2024-09-16T00:39:03.877836+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:39:03.877836+0000","last_undegraded":"2024-09-16T00:39:03.877836+0000","last_fullsized":"2024-09-16T00:39:03.877836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950389+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:39:03.950389+0000","last_peered":"2024-09-16T00:39:03.950389+0000","last_clean":"2024-09-16T00:39:03.950389+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:39:03.950389+0000","last_undegraded":"2024-09-16T00:39:03.950389+0000","last_fullsized":"2024-09-16T00:39:03.950389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.038107+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:39:04.038107+0000","last_peered":"2024-09-16T00:39:04.038107+0000","last_clean":"2024-09-16T00:39:04.038107+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:39:04.038107+0000","last_undegraded":"2024-09-16T00:39:04.038107+0000","last_fullsized":"2024-09-16T00:39:04.038107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'5","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:43.018995+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:39:43.018995+0000","last_peered":"2024-09-16T00:39:43.018995+0000","last_clean":"2024-09-16T00:39:43.018995+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:39:43.018995+0000","last_undegraded":"2024-09-16T00:39:43.018995+0000","last_fullsized":"2024-09-16T00:39:43.018995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037541+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:39:04.037541+0000","last_peered":"2024-09-16T00:39:04.037541+0000","last_clean":"2024-09-16T00:39:04.037541+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:39:04.037541+0000","last_undegraded":"2024-09-16T00:39:04.037541+0000","last_fullsized":"2024-09-16T00:39:04.037541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950097+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:39:03.950097+0000","last_peered":"2024-09-16T00:39:03.950097+0000","last_clean":"2024-09-16T00:39:03.950097+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:39:03.950097+0000","last_undegraded":"2024-09-16T00:39:03.950097+0000","last_fullsized":"2024-09-16T00:39:03.950097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951201+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:39:03.951201+0000","last_peered":"2024-09-16T00:39:03.951201+0000","last_clean":"2024-09-16T00:39:03.951201+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:39:03.951201+0000","last_undegraded":"2024-09-16T00:39:03.951201+0000","last_fullsized":"2024-09-16T00:39:03.951201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046477+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:39:03.046477+0000","last_peered":"2024-09-16T00:39:03.046477+0000","last_clean":"2024-09-16T00:39:03.046477+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:39:03.046477+0000","last_undegraded":"2024-09-16T00:39:03.046477+0000","last_fullsized":"2024-09-16T00:39:03.046477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950303+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:39:03.950303+0000","last_peered":"2024-09-16T00:39:03.950303+0000","last_clean":"2024-09-16T00:39:03.950303+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:39:03.950303+0000","last_undegraded":"2024-09-16T00:39:03.950303+0000","last_fullsized":"2024-09-16T00:39:03.950303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323726+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:39:03.323726+0000","last_peered":"2024-09-16T00:39:03.323726+0000","last_clean":"2024-09-16T00:39:03.323726+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:39:03.323726+0000","last_undegraded":"2024-09-16T00:39:03.323726+0000","last_fullsized":"2024-09-16T00:39:03.323726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950837+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:39:03.950837+0000","last_peered":"2024-09-16T00:39:03.950837+0000","last_clean":"2024-09-16T00:39:03.950837+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:39:03.950837+0000","last_undegraded":"2024-09-16T00:39:03.950837+0000","last_fullsized":"2024-09-16T00:39:03.950837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950288+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:39:03.950288+0000","last_peered":"2024-09-16T00:39:03.950288+0000","last_clean":"2024-09-16T00:39:03.950288+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:39:03.950288+0000","last_undegraded":"2024-09-16T00:39:03.950288+0000","last_fullsized":"2024-09-16T00:39:03.950288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046907+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:39:03.046907+0000","last_peered":"2024-09-16T00:39:03.046907+0000","last_clean":"2024-09-16T00:39:03.046907+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:39:03.046907+0000","last_undegraded":"2024-09-16T00:39:03.046907+0000","last_fullsized":"2024-09-16T00:39:03.046907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949991+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:39:03.949991+0000","last_peered":"2024-09-16T00:39:03.949991+0000","last_clean":"2024-09-16T00:39:03.949991+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:39:03.949991+0000","last_undegraded":"2024-09-16T00:39:03.949991+0000","last_fullsized":"2024-09-16T00:39:03.949991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950550+0000","last_change":"2024-09-16T00:38:52.972145+0000","last_active":"2024-09-16T00:39:03.950550+0000","last_peered":"2024-09-16T00:39:03.950550+0000","last_clean":"2024-09-16T00:39:03.950550+0000","last_became_active":"2024-09-16T00:38:52.971847+0000","last_became_peered":"2024-09-16T00:38:52.971847+0000","last_unstale":"2024-09-16T00:39:03.950550+0000","last_undegraded":"2024-09-16T00:39:03.950550+0000","last_fullsized":"2024-09-16T00:39:03.950550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:32.819389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949586+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:39:03.949586+0000","last_peered":"2024-09-16T00:39:03.949586+0000","last_clean":"2024-09-16T00:39:03.949586+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:39:03.949586+0000","last_undegraded":"2024-09-16T00:39:03.949586+0000","last_fullsized":"2024-09-16T00:39:03.949586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950106+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:39:03.950106+0000","last_peered":"2024-09-16T00:39:03.950106+0000","last_clean":"2024-09-16T00:39:03.950106+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:39:03.950106+0000","last_undegraded":"2024-09-16T00:39:03.950106+0000","last_fullsized":"2024-09-16T00:39:03.950106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949465+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:39:03.949465+0000","last_peered":"2024-09-16T00:39:03.949465+0000","last_clean":"2024-09-16T00:39:03.949465+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:39:03.949465+0000","last_undegraded":"2024-09-16T00:39:03.949465+0000","last_fullsized":"2024-09-16T00:39:03.949465+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046648+0000","last_change":"2024-09-16T00:38:50.875537+0000","last_active":"2024-09-16T00:39:03.046648+0000","last_peered":"2024-09-16T00:39:03.046648+0000","last_clean":"2024-09-16T00:39:03.046648+0000","last_became_active":"2024-09-16T00:38:50.875226+0000","last_became_peered":"2024-09-16T00:38:50.875226+0000","last_unstale":"2024-09-16T00:39:03.046648+0000","last_undegraded":"2024-09-16T00:39:03.046648+0000","last_fullsized":"2024-09-16T00:39:03.046648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:21.256432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951182+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:39:03.951182+0000","last_peered":"2024-09-16T00:39:03.951182+0000","last_clean":"2024-09-16T00:39:03.951182+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:39:03.951182+0000","last_undegraded":"2024-09-16T00:39:03.951182+0000","last_fullsized":"2024-09-16T00:39:03.951182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037837+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:39:04.037837+0000","last_peered":"2024-09-16T00:39:04.037837+0000","last_clean":"2024-09-16T00:39:04.037837+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:39:04.037837+0000","last_undegraded":"2024-09-16T00:39:04.037837+0000","last_fullsized":"2024-09-16T00:39:04.037837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877603+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:39:03.877603+0000","last_peered":"2024-09-16T00:39:03.877603+0000","last_clean":"2024-09-16T00:39:03.877603+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:39:03.877603+0000","last_undegraded":"2024-09-16T00:39:03.877603+0000","last_fullsized":"2024-09-16T00:39:03.877603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046796+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:39:03.046796+0000","last_peered":"2024-09-16T00:39:03.046796+0000","last_clean":"2024-09-16T00:39:03.046796+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:39:03.046796+0000","last_undegraded":"2024-09-16T00:39:03.046796+0000","last_fullsized":"2024-09-16T00:39:03.046796+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.527298+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:39:03.527298+0000","last_peered":"2024-09-16T00:39:03.527298+0000","last_clean":"2024-09-16T00:39:03.527298+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:39:03.527298+0000","last_undegraded":"2024-09-16T00:39:03.527298+0000","last_fullsized":"2024-09-16T00:39:03.527298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047393+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:39:03.047393+0000","last_peered":"2024-09-16T00:39:03.047393+0000","last_clean":"2024-09-16T00:39:03.047393+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:39:03.047393+0000","last_undegraded":"2024-09-16T00:39:03.047393+0000","last_fullsized":"2024-09-16T00:39:03.047393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950829+0000","last_change":"2024-09-16T00:38:51.423508+0000","last_active":"2024-09-16T00:39:03.950829+0000","last_peered":"2024-09-16T00:39:03.950829+0000","last_clean":"2024-09-16T00:39:03.950829+0000","last_became_active":"2024-09-16T00:38:51.423012+0000","last_became_peered":"2024-09-16T00:38:51.423012+0000","last_unstale":"2024-09-16T00:39:03.950829+0000","last_undegraded":"2024-09-16T00:39:03.950829+0000","last_fullsized":"2024-09-16T00:39:03.950829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:48.895796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576223+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:39:03.576223+0000","last_peered":"2024-09-16T00:39:03.576223+0000","last_clean":"2024-09-16T00:39:03.576223+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:39:03.576223+0000","last_undegraded":"2024-09-16T00:39:03.576223+0000","last_fullsized":"2024-09-16T00:39:03.576223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047184+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:39:03.047184+0000","last_peered":"2024-09-16T00:39:03.047184+0000","last_clean":"2024-09-16T00:39:03.047184+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:39:03.047184+0000","last_undegraded":"2024-09-16T00:39:03.047184+0000","last_fullsized":"2024-09-16T00:39:03.047184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.952197+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:39:03.952197+0000","last_peered":"2024-09-16T00:39:03.952197+0000","last_clean":"2024-09-16T00:39:03.952197+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:39:03.952197+0000","last_undegraded":"2024-09-16T00:39:03.952197+0000","last_fullsized":"2024-09-16T00:39:03.952197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045910+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:39:03.045910+0000","last_peered":"2024-09-16T00:39:03.045910+0000","last_clean":"2024-09-16T00:39:03.045910+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:39:03.045910+0000","last_undegraded":"2024-09-16T00:39:03.045910+0000","last_fullsized":"2024-09-16T00:39:03.045910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047257+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:39:03.047257+0000","last_peered":"2024-09-16T00:39:03.047257+0000","last_clean":"2024-09-16T00:39:03.047257+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:39:03.047257+0000","last_undegraded":"2024-09-16T00:39:03.047257+0000","last_fullsized":"2024-09-16T00:39:03.047257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950021+0000","last_change":"2024-09-16T00:38:50.875290+0000","last_active":"2024-09-16T00:39:03.950021+0000","last_peered":"2024-09-16T00:39:03.950021+0000","last_clean":"2024-09-16T00:39:03.950021+0000","last_became_active":"2024-09-16T00:38:50.874964+0000","last_became_peered":"2024-09-16T00:38:50.874964+0000","last_unstale":"2024-09-16T00:39:03.950021+0000","last_undegraded":"2024-09-16T00:39:03.950021+0000","last_fullsized":"2024-09-16T00:39:03.950021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:52.892725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950257+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:39:03.950257+0000","last_peered":"2024-09-16T00:39:03.950257+0000","last_clean":"2024-09-16T00:39:03.950257+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:39:03.950257+0000","last_undegraded":"2024-09-16T00:39:03.950257+0000","last_fullsized":"2024-09-16T00:39:03.950257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046776+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:39:03.046776+0000","last_peered":"2024-09-16T00:39:03.046776+0000","last_clean":"2024-09-16T00:39:03.046776+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:39:03.046776+0000","last_undegraded":"2024-09-16T00:39:03.046776+0000","last_fullsized":"2024-09-16T00:39:03.046776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951707+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:39:03.951707+0000","last_peered":"2024-09-16T00:39:03.951707+0000","last_clean":"2024-09-16T00:39:03.951707+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:39:03.951707+0000","last_undegraded":"2024-09-16T00:39:03.951707+0000","last_fullsized":"2024-09-16T00:39:03.951707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950428+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:39:03.950428+0000","last_peered":"2024-09-16T00:39:03.950428+0000","last_clean":"2024-09-16T00:39:03.950428+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:39:03.950428+0000","last_undegraded":"2024-09-16T00:39:03.950428+0000","last_fullsized":"2024-09-16T00:39:03.950428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950603+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:39:03.950603+0000","last_peered":"2024-09-16T00:39:03.950603+0000","last_clean":"2024-09-16T00:39:03.950603+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:39:03.950603+0000","last_undegraded":"2024-09-16T00:39:03.950603+0000","last_fullsized":"2024-09-16T00:39:03.950603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950466+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:39:03.950466+0000","last_peered":"2024-09-16T00:39:03.950466+0000","last_clean":"2024-09-16T00:39:03.950466+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:39:03.950466+0000","last_undegraded":"2024-09-16T00:39:03.950466+0000","last_fullsized":"2024-09-16T00:39:03.950466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950077+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:39:03.950077+0000","last_peered":"2024-09-16T00:39:03.950077+0000","last_clean":"2024-09-16T00:39:03.950077+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:39:03.950077+0000","last_undegraded":"2024-09-16T00:39:03.950077+0000","last_fullsized":"2024-09-16T00:39:03.950077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046570+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:39:03.046570+0000","last_peered":"2024-09-16T00:39:03.046570+0000","last_clean":"2024-09-16T00:39:03.046570+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:39:03.046570+0000","last_undegraded":"2024-09-16T00:39:03.046570+0000","last_fullsized":"2024-09-16T00:39:03.046570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877280+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:39:03.877280+0000","last_peered":"2024-09-16T00:39:03.877280+0000","last_clean":"2024-09-16T00:39:03.877280+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:39:03.877280+0000","last_undegraded":"2024-09-16T00:39:03.877280+0000","last_fullsized":"2024-09-16T00:39:03.877280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323969+0000","last_change":"2024-09-16T00:39:00.049744+0000","last_active":"2024-09-16T00:39:03.323969+0000","last_peered":"2024-09-16T00:39:03.323969+0000","last_clean":"2024-09-16T00:39:03.323969+0000","last_became_active":"2024-09-16T00:39:00.048919+0000","last_became_peered":"2024-09-16T00:39:00.048919+0000","last_unstale":"2024-09-16T00:39:03.323969+0000","last_undegraded":"2024-09-16T00:39:03.323969+0000","last_fullsized":"2024-09-16T00:39:03.323969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:49:19.509215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047002+0000","last_change":"2024-09-16T00:38:52.974330+0000","last_active":"2024-09-16T00:39:03.047002+0000","last_peered":"2024-09-16T00:39:03.047002+0000","last_clean":"2024-09-16T00:39:03.047002+0000","last_became_active":"2024-09-16T00:38:52.974004+0000","last_became_peered":"2024-09-16T00:38:52.974004+0000","last_unstale":"2024-09-16T00:39:03.047002+0000","last_undegraded":"2024-09-16T00:39:03.047002+0000","last_fullsized":"2024-09-16T00:39:03.047002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:31.820720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950818+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:39:03.950818+0000","last_peered":"2024-09-16T00:39:03.950818+0000","last_clean":"2024-09-16T00:39:03.950818+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:39:03.950818+0000","last_undegraded":"2024-09-16T00:39:03.950818+0000","last_fullsized":"2024-09-16T00:39:03.950818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046458+0000","last_change":"2024-09-16T00:39:00.053518+0000","last_active":"2024-09-16T00:39:03.046458+0000","last_peered":"2024-09-16T00:39:03.046458+0000","last_clean":"2024-09-16T00:39:03.046458+0000","last_became_active":"2024-09-16T00:39:00.052592+0000","last_became_peered":"2024-09-16T00:39:00.052592+0000","last_unstale":"2024-09-16T00:39:03.046458+0000","last_undegraded":"2024-09-16T00:39:03.046458+0000","last_fullsized":"2024-09-16T00:39:03.046458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:07:58.151992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949662+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:39:03.949662+0000","last_peered":"2024-09-16T00:39:03.949662+0000","last_clean":"2024-09-16T00:39:03.949662+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:39:03.949662+0000","last_undegraded":"2024-09-16T00:39:03.949662+0000","last_fullsized":"2024-09-16T00:39:03.949662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950664+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:39:03.950664+0000","last_peered":"2024-09-16T00:39:03.950664+0000","last_clean":"2024-09-16T00:39:03.950664+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:39:03.950664+0000","last_undegraded":"2024-09-16T00:39:03.950664+0000","last_fullsized":"2024-09-16T00:39:03.950664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951745+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:39:03.951745+0000","last_peered":"2024-09-16T00:39:03.951745+0000","last_clean":"2024-09-16T00:39:03.951745+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:39:03.951745+0000","last_undegraded":"2024-09-16T00:39:03.951745+0000","last_fullsized":"2024-09-16T00:39:03.951745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950700+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:39:03.950700+0000","last_peered":"2024-09-16T00:39:03.950700+0000","last_clean":"2024-09-16T00:39:03.950700+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:39:03.950700+0000","last_undegraded":"2024-09-16T00:39:03.950700+0000","last_fullsized":"2024-09-16T00:39:03.950700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'24","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:45.161142+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:39:45.161142+0000","last_peered":"2024-09-16T00:39:45.161142+0000","last_clean":"2024-09-16T00:39:45.161142+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:39:45.161142+0000","last_undegraded":"2024-09-16T00:39:45.161142+0000","last_fullsized":"2024-09-16T00:39:45.161142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":62,"num_write":24,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323462+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:39:03.323462+0000","last_peered":"2024-09-16T00:39:03.323462+0000","last_clean":"2024-09-16T00:39:03.323462+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:39:03.323462+0000","last_undegraded":"2024-09-16T00:39:03.323462+0000","last_fullsized":"2024-09-16T00:39:03.323462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576351+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:39:03.576351+0000","last_peered":"2024-09-16T00:39:03.576351+0000","last_clean":"2024-09-16T00:39:03.576351+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:39:03.576351+0000","last_undegraded":"2024-09-16T00:39:03.576351+0000","last_fullsized":"2024-09-16T00:39:03.576351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.952036+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:39:03.952036+0000","last_peered":"2024-09-16T00:39:03.952036+0000","last_clean":"2024-09-16T00:39:03.952036+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:39:03.952036+0000","last_undegraded":"2024-09-16T00:39:03.952036+0000","last_fullsized":"2024-09-16T00:39:03.952036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877436+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:39:03.877436+0000","last_peered":"2024-09-16T00:39:03.877436+0000","last_clean":"2024-09-16T00:39:03.877436+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:39:03.877436+0000","last_undegraded":"2024-09-16T00:39:03.877436+0000","last_fullsized":"2024-09-16T00:39:03.877436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046131+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:39:03.046131+0000","last_peered":"2024-09-16T00:39:03.046131+0000","last_clean":"2024-09-16T00:39:03.046131+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:39:03.046131+0000","last_undegraded":"2024-09-16T00:39:03.046131+0000","last_fullsized":"2024-09-16T00:39:03.046131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047184+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:39:03.047184+0000","last_peered":"2024-09-16T00:39:03.047184+0000","last_clean":"2024-09-16T00:39:03.047184+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:39:03.047184+0000","last_undegraded":"2024-09-16T00:39:03.047184+0000","last_fullsized":"2024-09-16T00:39:03.047184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323310+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:39:03.323310+0000","last_peered":"2024-09-16T00:39:03.323310+0000","last_clean":"2024-09-16T00:39:03.323310+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:39:03.323310+0000","last_undegraded":"2024-09-16T00:39:03.323310+0000","last_fullsized":"2024-09-16T00:39:03.323310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576575+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:39:03.576575+0000","last_peered":"2024-09-16T00:39:03.576575+0000","last_clean":"2024-09-16T00:39:03.576575+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:39:03.576575+0000","last_undegraded":"2024-09-16T00:39:03.576575+0000","last_fullsized":"2024-09-16T00:39:03.576575+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950559+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:39:03.950559+0000","last_peered":"2024-09-16T00:39:03.950559+0000","last_clean":"2024-09-16T00:39:03.950559+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:39:03.950559+0000","last_undegraded":"2024-09-16T00:39:03.950559+0000","last_fullsized":"2024-09-16T00:39:03.950559+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037706+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:39:04.037706+0000","last_peered":"2024-09-16T00:39:04.037706+0000","last_clean":"2024-09-16T00:39:04.037706+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:39:04.037706+0000","last_undegraded":"2024-09-16T00:39:04.037706+0000","last_fullsized":"2024-09-16T00:39:04.037706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045949+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:39:03.045949+0000","last_peered":"2024-09-16T00:39:03.045949+0000","last_clean":"2024-09-16T00:39:03.045949+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:39:03.045949+0000","last_undegraded":"2024-09-16T00:39:03.045949+0000","last_fullsized":"2024-09-16T00:39:03.045949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323357+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:39:03.323357+0000","last_peered":"2024-09-16T00:39:03.323357+0000","last_clean":"2024-09-16T00:39:03.323357+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:39:03.323357+0000","last_undegraded":"2024-09-16T00:39:03.323357+0000","last_fullsized":"2024-09-16T00:39:03.323357+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950717+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:39:03.950717+0000","last_peered":"2024-09-16T00:39:03.950717+0000","last_clean":"2024-09-16T00:39:03.950717+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:39:03.950717+0000","last_undegraded":"2024-09-16T00:39:03.950717+0000","last_fullsized":"2024-09-16T00:39:03.950717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046805+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:39:03.046805+0000","last_peered":"2024-09-16T00:39:03.046805+0000","last_clean":"2024-09-16T00:39:03.046805+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:39:03.046805+0000","last_undegraded":"2024-09-16T00:39:03.046805+0000","last_fullsized":"2024-09-16T00:39:03.046805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037643+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:39:04.037643+0000","last_peered":"2024-09-16T00:39:04.037643+0000","last_clean":"2024-09-16T00:39:04.037643+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:39:04.037643+0000","last_undegraded":"2024-09-16T00:39:04.037643+0000","last_fullsized":"2024-09-16T00:39:04.037643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949465+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:39:03.949465+0000","last_peered":"2024-09-16T00:39:03.949465+0000","last_clean":"2024-09-16T00:39:03.949465+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:39:03.949465+0000","last_undegraded":"2024-09-16T00:39:03.949465+0000","last_fullsized":"2024-09-16T00:39:03.949465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950221+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:39:03.950221+0000","last_peered":"2024-09-16T00:39:03.950221+0000","last_clean":"2024-09-16T00:39:03.950221+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:39:03.950221+0000","last_undegraded":"2024-09-16T00:39:03.950221+0000","last_fullsized":"2024-09-16T00:39:03.950221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951038+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:39:03.951038+0000","last_peered":"2024-09-16T00:39:03.951038+0000","last_clean":"2024-09-16T00:39:03.951038+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:39:03.951038+0000","last_undegraded":"2024-09-16T00:39:03.951038+0000","last_fullsized":"2024-09-16T00:39:03.951038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950457+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:39:03.950457+0000","last_peered":"2024-09-16T00:39:03.950457+0000","last_clean":"2024-09-16T00:39:03.950457+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:39:03.950457+0000","last_undegraded":"2024-09-16T00:39:03.950457+0000","last_fullsized":"2024-09-16T00:39:03.950457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951145+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:39:03.951145+0000","last_peered":"2024-09-16T00:39:03.951145+0000","last_clean":"2024-09-16T00:39:03.951145+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:39:03.951145+0000","last_undegraded":"2024-09-16T00:39:03.951145+0000","last_fullsized":"2024-09-16T00:39:03.951145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951025+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:39:03.951025+0000","last_peered":"2024-09-16T00:39:03.951025+0000","last_clean":"2024-09-16T00:39:03.951025+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:39:03.951025+0000","last_undegraded":"2024-09-16T00:39:03.951025+0000","last_fullsized":"2024-09-16T00:39:03.951025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951108+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:39:03.951108+0000","last_peered":"2024-09-16T00:39:03.951108+0000","last_clean":"2024-09-16T00:39:03.951108+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:39:03.951108+0000","last_undegraded":"2024-09-16T00:39:03.951108+0000","last_fullsized":"2024-09-16T00:39:03.951108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046274+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:39:03.046274+0000","last_peered":"2024-09-16T00:39:03.046274+0000","last_clean":"2024-09-16T00:39:03.046274+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:39:03.046274+0000","last_undegraded":"2024-09-16T00:39:03.046274+0000","last_fullsized":"2024-09-16T00:39:03.046274+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950637+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:39:03.950637+0000","last_peered":"2024-09-16T00:39:03.950637+0000","last_clean":"2024-09-16T00:39:03.950637+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:39:03.950637+0000","last_undegraded":"2024-09-16T00:39:03.950637+0000","last_fullsized":"2024-09-16T00:39:03.950637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950140+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:39:03.950140+0000","last_peered":"2024-09-16T00:39:03.950140+0000","last_clean":"2024-09-16T00:39:03.950140+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:39:03.950140+0000","last_undegraded":"2024-09-16T00:39:03.950140+0000","last_fullsized":"2024-09-16T00:39:03.950140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045819+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:39:03.045819+0000","last_peered":"2024-09-16T00:39:03.045819+0000","last_clean":"2024-09-16T00:39:03.045819+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:39:03.045819+0000","last_undegraded":"2024-09-16T00:39:03.045819+0000","last_fullsized":"2024-09-16T00:39:03.045819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950024+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:39:03.950024+0000","last_peered":"2024-09-16T00:39:03.950024+0000","last_clean":"2024-09-16T00:39:03.950024+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:39:03.950024+0000","last_undegraded":"2024-09-16T00:39:03.950024+0000","last_fullsized":"2024-09-16T00:39:03.950024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046730+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:39:03.046730+0000","last_peered":"2024-09-16T00:39:03.046730+0000","last_clean":"2024-09-16T00:39:03.046730+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:39:03.046730+0000","last_undegraded":"2024-09-16T00:39:03.046730+0000","last_fullsized":"2024-09-16T00:39:03.046730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950908+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:39:03.950908+0000","last_peered":"2024-09-16T00:39:03.950908+0000","last_clean":"2024-09-16T00:39:03.950908+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:39:03.950908+0000","last_undegraded":"2024-09-16T00:39:03.950908+0000","last_fullsized":"2024-09-16T00:39:03.950908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037971+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:39:04.037971+0000","last_peered":"2024-09-16T00:39:04.037971+0000","last_clean":"2024-09-16T00:39:04.037971+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:39:04.037971+0000","last_undegraded":"2024-09-16T00:39:04.037971+0000","last_fullsized":"2024-09-16T00:39:04.037971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950443+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:39:03.950443+0000","last_peered":"2024-09-16T00:39:03.950443+0000","last_clean":"2024-09-16T00:39:03.950443+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:39:03.950443+0000","last_undegraded":"2024-09-16T00:39:03.950443+0000","last_fullsized":"2024-09-16T00:39:03.950443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037733+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:39:04.037733+0000","last_peered":"2024-09-16T00:39:04.037733+0000","last_clean":"2024-09-16T00:39:04.037733+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:39:04.037733+0000","last_undegraded":"2024-09-16T00:39:04.037733+0000","last_fullsized":"2024-09-16T00:39:04.037733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576251+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:39:03.576251+0000","last_peered":"2024-09-16T00:39:03.576251+0000","last_clean":"2024-09-16T00:39:03.576251+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:39:03.576251+0000","last_undegraded":"2024-09-16T00:39:03.576251+0000","last_fullsized":"2024-09-16T00:39:03.576251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949846+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:39:03.949846+0000","last_peered":"2024-09-16T00:39:03.949846+0000","last_clean":"2024-09-16T00:39:03.949846+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:39:03.949846+0000","last_undegraded":"2024-09-16T00:39:03.949846+0000","last_fullsized":"2024-09-16T00:39:03.949846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046394+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:39:03.046394+0000","last_peered":"2024-09-16T00:39:03.046394+0000","last_clean":"2024-09-16T00:39:03.046394+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:39:03.046394+0000","last_undegraded":"2024-09-16T00:39:03.046394+0000","last_fullsized":"2024-09-16T00:39:03.046394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951268+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:39:03.951268+0000","last_peered":"2024-09-16T00:39:03.951268+0000","last_clean":"2024-09-16T00:39:03.951268+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:39:03.951268+0000","last_undegraded":"2024-09-16T00:39:03.951268+0000","last_fullsized":"2024-09-16T00:39:03.951268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323891+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:39:03.323891+0000","last_peered":"2024-09-16T00:39:03.323891+0000","last_clean":"2024-09-16T00:39:03.323891+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:39:03.323891+0000","last_undegraded":"2024-09-16T00:39:03.323891+0000","last_fullsized":"2024-09-16T00:39:03.323891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046967+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:39:03.046967+0000","last_peered":"2024-09-16T00:39:03.046967+0000","last_clean":"2024-09-16T00:39:03.046967+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:39:03.046967+0000","last_undegraded":"2024-09-16T00:39:03.046967+0000","last_fullsized":"2024-09-16T00:39:03.046967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.575931+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:39:03.575931+0000","last_peered":"2024-09-16T00:39:03.575931+0000","last_clean":"2024-09-16T00:39:03.575931+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:39:03.575931+0000","last_undegraded":"2024-09-16T00:39:03.575931+0000","last_fullsized":"2024-09-16T00:39:03.575931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045615+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:39:03.045615+0000","last_peered":"2024-09-16T00:39:03.045615+0000","last_clean":"2024-09-16T00:39:03.045615+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:39:03.045615+0000","last_undegraded":"2024-09-16T00:39:03.045615+0000","last_fullsized":"2024-09-16T00:39:03.045615+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045626+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:39:03.045626+0000","last_peered":"2024-09-16T00:39:03.045626+0000","last_clean":"2024-09-16T00:39:03.045626+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:39:03.045626+0000","last_undegraded":"2024-09-16T00:39:03.045626+0000","last_fullsized":"2024-09-16T00:39:03.045626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949809+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:39:03.949809+0000","last_peered":"2024-09-16T00:39:03.949809+0000","last_clean":"2024-09-16T00:39:03.949809+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:39:03.949809+0000","last_undegraded":"2024-09-16T00:39:03.949809+0000","last_fullsized":"2024-09-16T00:39:03.949809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877612+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:39:03.877612+0000","last_peered":"2024-09-16T00:39:03.877612+0000","last_clean":"2024-09-16T00:39:03.877612+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:39:03.877612+0000","last_undegraded":"2024-09-16T00:39:03.877612+0000","last_fullsized":"2024-09-16T00:39:03.877612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047387+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:39:03.047387+0000","last_peered":"2024-09-16T00:39:03.047387+0000","last_clean":"2024-09-16T00:39:03.047387+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:39:03.047387+0000","last_undegraded":"2024-09-16T00:39:03.047387+0000","last_fullsized":"2024-09-16T00:39:03.047387+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950638+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:39:03.950638+0000","last_peered":"2024-09-16T00:39:03.950638+0000","last_clean":"2024-09-16T00:39:03.950638+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:39:03.950638+0000","last_undegraded":"2024-09-16T00:39:03.950638+0000","last_fullsized":"2024-09-16T00:39:03.950638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576567+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:39:03.576567+0000","last_peered":"2024-09-16T00:39:03.576567+0000","last_clean":"2024-09-16T00:39:03.576567+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:39:03.576567+0000","last_undegraded":"2024-09-16T00:39:03.576567+0000","last_fullsized":"2024-09-16T00:39:03.576567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037546+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:39:04.037546+0000","last_peered":"2024-09-16T00:39:04.037546+0000","last_clean":"2024-09-16T00:39:04.037546+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:39:04.037546+0000","last_undegraded":"2024-09-16T00:39:04.037546+0000","last_fullsized":"2024-09-16T00:39:04.037546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046655+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:39:03.046655+0000","last_peered":"2024-09-16T00:39:03.046655+0000","last_clean":"2024-09-16T00:39:03.046655+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:39:03.046655+0000","last_undegraded":"2024-09-16T00:39:03.046655+0000","last_fullsized":"2024-09-16T00:39:03.046655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950213+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:39:03.950213+0000","last_peered":"2024-09-16T00:39:03.950213+0000","last_clean":"2024-09-16T00:39:03.950213+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:39:03.950213+0000","last_undegraded":"2024-09-16T00:39:03.950213+0000","last_fullsized":"2024-09-16T00:39:03.950213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950858+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:39:03.950858+0000","last_peered":"2024-09-16T00:39:03.950858+0000","last_clean":"2024-09-16T00:39:03.950858+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:39:03.950858+0000","last_undegraded":"2024-09-16T00:39:03.950858+0000","last_fullsized":"2024-09-16T00:39:03.950858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877571+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:39:03.877571+0000","last_peered":"2024-09-16T00:39:03.877571+0000","last_clean":"2024-09-16T00:39:03.877571+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:39:03.877571+0000","last_undegraded":"2024-09-16T00:39:03.877571+0000","last_fullsized":"2024-09-16T00:39:03.877571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045968+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:39:03.045968+0000","last_peered":"2024-09-16T00:39:03.045968+0000","last_clean":"2024-09-16T00:39:03.045968+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:39:03.045968+0000","last_undegraded":"2024-09-16T00:39:03.045968+0000","last_fullsized":"2024-09-16T00:39:03.045968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045904+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:39:03.045904+0000","last_peered":"2024-09-16T00:39:03.045904+0000","last_clean":"2024-09-16T00:39:03.045904+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:39:03.045904+0000","last_undegraded":"2024-09-16T00:39:03.045904+0000","last_fullsized":"2024-09-16T00:39:03.045904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046933+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:39:03.046933+0000","last_peered":"2024-09-16T00:39:03.046933+0000","last_clean":"2024-09-16T00:39:03.046933+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:39:03.046933+0000","last_undegraded":"2024-09-16T00:39:03.046933+0000","last_fullsized":"2024-09-16T00:39:03.046933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49896,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":35,"num_read_kb":68,"num_write":71,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":149688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547224,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":480695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1630208,"data_stored":1116259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939681,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":485373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168615,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":481299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1572864,"data_stored":1075741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":476599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1071245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":480717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":476703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":525731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":525697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":970752,"data_stored":476689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":49132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":49292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:39:50.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:39:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:50 smithi081 ceph-mon[35539]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:39:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:50 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:50 smithi102 ceph-mon[35364]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:39:50.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:50 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:50.844 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:51.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:51 smithi031 ceph-mon[30956]: pgmap v313: 129 pgs: 129 active+clean; 625 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-16T00:39:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 -- 172.21.15.31:0/2297733423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 msgr2=0x7f4cfc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 --2- 172.21.15.31:0/2297733423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 0x7f4cfc1067e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009900 tx=0x7f4cec02a410 comp rx=0 tx=0).stop 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 -- 172.21.15.31:0/2297733423 shutdown_connections 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 --2- 172.21.15.31:0/2297733423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4cfc106f10 0x7f4cfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 --2- 172.21.15.31:0/2297733423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 0x7f4cfc1067e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 --2- 172.21.15.31:0/2297733423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cfc101de0 0x7f4cfc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 -- 172.21.15.31:0/2297733423 >> 172.21.15.31:0/2297733423 conn(0x7f4cfc0fd8c0 msgr2=0x7f4cfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 -- 172.21.15.31:0/2297733423 shutdown_connections 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 -- 172.21.15.31:0/2297733423 wait complete. 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 Processor -- start 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.346+0000 7f4d047e8640 1 -- start start 2024-09-16T00:39:51.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d047e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d047e8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 0x7f4cfc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d047e8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d047e8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc111f30 con 0x7f4cfc102790 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d047e8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc111db0 con 0x7f4cfc101de0 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d047e8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1120b0 con 0x7f4cfc106f10 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d01d83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 0x7f4cfc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d01d83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 0x7f4cfc198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54198/0 (socket says 172.21.15.31:54198) 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d01d83640 1 -- 172.21.15.31:0/4065188540 learned_addr learned my addr 172.21.15.31:0/4065188540 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d02584640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d02d85640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d01d83640 1 -- 172.21.15.31:0/4065188540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4cfc106f10 msgr2=0x7f4cfc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d01d83640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d01d83640 1 -- 172.21.15.31:0/4065188540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cfc101de0 msgr2=0x7f4cfc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d01d83640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d01d83640 1 -- 172.21.15.31:0/4065188540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc19d3e0 con 0x7f4cfc102790 2024-09-16T00:39:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d02d85640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.347+0000 7f4d01d83640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 0x7f4cfc198970 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4cec006010 tx=0x7f4cec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.348+0000 7f4cf37fe640 1 -- 172.21.15.31:0/4065188540 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec03f030 con 0x7f4cfc102790 2024-09-16T00:39:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.348+0000 7f4cf37fe640 1 -- 172.21.15.31:0/4065188540 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cec003ce0 con 0x7f4cfc102790 2024-09-16T00:39:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.348+0000 7f4cf37fe640 1 -- 172.21.15.31:0/4065188540 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec035070 con 0x7f4cfc102790 2024-09-16T00:39:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.348+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cfc19d670 con 0x7f4cfc102790 2024-09-16T00:39:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.348+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cfc1a4fb0 con 0x7f4cfc102790 2024-09-16T00:39:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.349+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cfc1021c0 con 0x7f4cfc102790 2024-09-16T00:39:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.350+0000 7f4cf37fe640 1 -- 172.21.15.31:0/4065188540 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4cec042030 con 0x7f4cfc102790 2024-09-16T00:39:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.350+0000 7f4cf37fe640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ce80a3350 0x7f4ce80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.351+0000 7f4d02584640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ce80a3350 0x7f4ce80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.351+0000 7f4cf37fe640 1 -- 172.21.15.31:0/4065188540 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f4cec0f0270 con 0x7f4cfc102790 2024-09-16T00:39:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.351+0000 7f4d02584640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ce80a3350 0x7f4ce80a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800bbe0 tx=0x7f4cf800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.355+0000 7f4cf37fe640 1 -- 172.21.15.31:0/4065188540 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4cec0bccb0 con 0x7f4cfc102790 2024-09-16T00:39:51.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:51 smithi081 ceph-mon[35539]: pgmap v313: 129 pgs: 129 active+clean; 625 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-16T00:39:51.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:51 smithi102 ceph-mon[35364]: pgmap v313: 129 pgs: 129 active+clean; 625 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-16T00:39:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.583+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4cfc062630 con 0x7f4ce80a3350 2024-09-16T00:39:51.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.592+0000 7f4cf37fe640 1 -- 172.21.15.31:0/4065188540 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340256 (secure 0 0 0) 0x7f4cfc062630 con 0x7f4ce80a3350 2024-09-16T00:39:51.598 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:51.600 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:39:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ce80a3350 msgr2=0x7f4ce80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ce80a3350 0x7f4ce80a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800bbe0 tx=0x7f4cf800b390 comp rx=0 tx=0).stop 2024-09-16T00:39:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 msgr2=0x7f4cfc198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 0x7f4cfc198970 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4cec006010 tx=0x7f4cec0042f0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 shutdown_connections 2024-09-16T00:39:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4ce80a3350 0x7f4ce80a5810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4cfc102790 0x7f4cfc198970 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 --2- 172.21.15.31:0/4065188540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 >> 172.21.15.31:0/4065188540 conn(0x7f4cfc0fd8c0 msgr2=0x7f4cfc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 shutdown_connections 2024-09-16T00:39:51.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:51.596+0000 7f4d047e8640 1 -- 172.21.15.31:0/4065188540 wait complete. 2024-09-16T00:39:52.333 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-16T00:39:49.664858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640498,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":161,"num_read_kb":177,"num_write":260,"num_write_kb":3014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400028,"kb_used_data":13328,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541796,"statfs":{"total":1151940427776,"available":1151530799104,"internally_reserved":0,"allocated":13647872,"data_stored":7672748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45488,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":58,"num_write":30,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003449"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.324102+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:39:03.324102+0000","last_peered":"2024-09-16T00:39:03.324102+0000","last_clean":"2024-09-16T00:39:03.324102+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:39:03.324102+0000","last_undegraded":"2024-09-16T00:39:03.324102+0000","last_fullsized":"2024-09-16T00:39:03.324102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037896+0000","last_change":"2024-09-16T00:38:52.968681+0000","last_active":"2024-09-16T00:39:04.037896+0000","last_peered":"2024-09-16T00:39:04.037896+0000","last_clean":"2024-09-16T00:39:04.037896+0000","last_became_active":"2024-09-16T00:38:52.968359+0000","last_became_peered":"2024-09-16T00:38:52.968359+0000","last_unstale":"2024-09-16T00:39:04.037896+0000","last_undegraded":"2024-09-16T00:39:04.037896+0000","last_fullsized":"2024-09-16T00:39:04.037896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:17.870176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950134+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:39:03.950134+0000","last_peered":"2024-09-16T00:39:03.950134+0000","last_clean":"2024-09-16T00:39:03.950134+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:39:03.950134+0000","last_undegraded":"2024-09-16T00:39:03.950134+0000","last_fullsized":"2024-09-16T00:39:03.950134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.038136+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:39:04.038136+0000","last_peered":"2024-09-16T00:39:04.038136+0000","last_clean":"2024-09-16T00:39:04.038136+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:39:04.038136+0000","last_undegraded":"2024-09-16T00:39:04.038136+0000","last_fullsized":"2024-09-16T00:39:04.038136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046126+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:39:03.046126+0000","last_peered":"2024-09-16T00:39:03.046126+0000","last_clean":"2024-09-16T00:39:03.046126+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:39:03.046126+0000","last_undegraded":"2024-09-16T00:39:03.046126+0000","last_fullsized":"2024-09-16T00:39:03.046126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877021+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:39:03.877021+0000","last_peered":"2024-09-16T00:39:03.877021+0000","last_clean":"2024-09-16T00:39:03.877021+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:39:03.877021+0000","last_undegraded":"2024-09-16T00:39:03.877021+0000","last_fullsized":"2024-09-16T00:39:03.877021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950908+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:39:03.950908+0000","last_peered":"2024-09-16T00:39:03.950908+0000","last_clean":"2024-09-16T00:39:03.950908+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:39:03.950908+0000","last_undegraded":"2024-09-16T00:39:03.950908+0000","last_fullsized":"2024-09-16T00:39:03.950908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576217+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:39:03.576217+0000","last_peered":"2024-09-16T00:39:03.576217+0000","last_clean":"2024-09-16T00:39:03.576217+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:39:03.576217+0000","last_undegraded":"2024-09-16T00:39:03.576217+0000","last_fullsized":"2024-09-16T00:39:03.576217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877200+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:39:03.877200+0000","last_peered":"2024-09-16T00:39:03.877200+0000","last_clean":"2024-09-16T00:39:03.877200+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:39:03.877200+0000","last_undegraded":"2024-09-16T00:39:03.877200+0000","last_fullsized":"2024-09-16T00:39:03.877200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951792+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:39:03.951792+0000","last_peered":"2024-09-16T00:39:03.951792+0000","last_clean":"2024-09-16T00:39:03.951792+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:39:03.951792+0000","last_undegraded":"2024-09-16T00:39:03.951792+0000","last_fullsized":"2024-09-16T00:39:03.951792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951852+0000","last_change":"2024-09-16T00:38:52.975634+0000","last_active":"2024-09-16T00:39:03.951852+0000","last_peered":"2024-09-16T00:39:03.951852+0000","last_clean":"2024-09-16T00:39:03.951852+0000","last_became_active":"2024-09-16T00:38:52.975314+0000","last_became_peered":"2024-09-16T00:38:52.975314+0000","last_unstale":"2024-09-16T00:39:03.951852+0000","last_undegraded":"2024-09-16T00:39:03.951852+0000","last_fullsized":"2024-09-16T00:39:03.951852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:28.716600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877164+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:39:03.877164+0000","last_peered":"2024-09-16T00:39:03.877164+0000","last_clean":"2024-09-16T00:39:03.877164+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:39:03.877164+0000","last_undegraded":"2024-09-16T00:39:03.877164+0000","last_fullsized":"2024-09-16T00:39:03.877164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046796+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:39:03.046796+0000","last_peered":"2024-09-16T00:39:03.046796+0000","last_clean":"2024-09-16T00:39:03.046796+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:39:03.046796+0000","last_undegraded":"2024-09-16T00:39:03.046796+0000","last_fullsized":"2024-09-16T00:39:03.046796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951025+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:39:03.951025+0000","last_peered":"2024-09-16T00:39:03.951025+0000","last_clean":"2024-09-16T00:39:03.951025+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:39:03.951025+0000","last_undegraded":"2024-09-16T00:39:03.951025+0000","last_fullsized":"2024-09-16T00:39:03.951025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877553+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:39:03.877553+0000","last_peered":"2024-09-16T00:39:03.877553+0000","last_clean":"2024-09-16T00:39:03.877553+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:39:03.877553+0000","last_undegraded":"2024-09-16T00:39:03.877553+0000","last_fullsized":"2024-09-16T00:39:03.877553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037967+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:39:04.037967+0000","last_peered":"2024-09-16T00:39:04.037967+0000","last_clean":"2024-09-16T00:39:04.037967+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:39:04.037967+0000","last_undegraded":"2024-09-16T00:39:04.037967+0000","last_fullsized":"2024-09-16T00:39:04.037967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950267+0000","last_change":"2024-09-16T00:38:50.874884+0000","last_active":"2024-09-16T00:39:03.950267+0000","last_peered":"2024-09-16T00:39:03.950267+0000","last_clean":"2024-09-16T00:39:03.950267+0000","last_became_active":"2024-09-16T00:38:50.874466+0000","last_became_peered":"2024-09-16T00:38:50.874466+0000","last_unstale":"2024-09-16T00:39:03.950267+0000","last_undegraded":"2024-09-16T00:39:03.950267+0000","last_fullsized":"2024-09-16T00:39:03.950267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:35.354581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877487+0000","last_change":"2024-09-16T00:39:00.047867+0000","last_active":"2024-09-16T00:39:03.877487+0000","last_peered":"2024-09-16T00:39:03.877487+0000","last_clean":"2024-09-16T00:39:03.877487+0000","last_became_active":"2024-09-16T00:39:00.047436+0000","last_became_peered":"2024-09-16T00:39:00.047436+0000","last_unstale":"2024-09-16T00:39:03.877487+0000","last_undegraded":"2024-09-16T00:39:03.877487+0000","last_fullsized":"2024-09-16T00:39:03.877487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:04.133052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046350+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:39:03.046350+0000","last_peered":"2024-09-16T00:39:03.046350+0000","last_clean":"2024-09-16T00:39:03.046350+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:39:03.046350+0000","last_undegraded":"2024-09-16T00:39:03.046350+0000","last_fullsized":"2024-09-16T00:39:03.046350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037850+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:39:04.037850+0000","last_peered":"2024-09-16T00:39:04.037850+0000","last_clean":"2024-09-16T00:39:04.037850+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:39:04.037850+0000","last_undegraded":"2024-09-16T00:39:04.037850+0000","last_fullsized":"2024-09-16T00:39:04.037850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323966+0000","last_change":"2024-09-16T00:39:00.053720+0000","last_active":"2024-09-16T00:39:03.323966+0000","last_peered":"2024-09-16T00:39:03.323966+0000","last_clean":"2024-09-16T00:39:03.323966+0000","last_became_active":"2024-09-16T00:39:00.053309+0000","last_became_peered":"2024-09-16T00:39:00.053309+0000","last_unstale":"2024-09-16T00:39:03.323966+0000","last_undegraded":"2024-09-16T00:39:03.323966+0000","last_fullsized":"2024-09-16T00:39:03.323966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:07.390804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951256+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:39:03.951256+0000","last_peered":"2024-09-16T00:39:03.951256+0000","last_clean":"2024-09-16T00:39:03.951256+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:39:03.951256+0000","last_undegraded":"2024-09-16T00:39:03.951256+0000","last_fullsized":"2024-09-16T00:39:03.951256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576050+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:39:03.576050+0000","last_peered":"2024-09-16T00:39:03.576050+0000","last_clean":"2024-09-16T00:39:03.576050+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:39:03.576050+0000","last_undegraded":"2024-09-16T00:39:03.576050+0000","last_fullsized":"2024-09-16T00:39:03.576050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877507+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:39:03.877507+0000","last_peered":"2024-09-16T00:39:03.877507+0000","last_clean":"2024-09-16T00:39:03.877507+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:39:03.877507+0000","last_undegraded":"2024-09-16T00:39:03.877507+0000","last_fullsized":"2024-09-16T00:39:03.877507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046022+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:39:03.046022+0000","last_peered":"2024-09-16T00:39:03.046022+0000","last_clean":"2024-09-16T00:39:03.046022+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:39:03.046022+0000","last_undegraded":"2024-09-16T00:39:03.046022+0000","last_fullsized":"2024-09-16T00:39:03.046022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877836+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:39:03.877836+0000","last_peered":"2024-09-16T00:39:03.877836+0000","last_clean":"2024-09-16T00:39:03.877836+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:39:03.877836+0000","last_undegraded":"2024-09-16T00:39:03.877836+0000","last_fullsized":"2024-09-16T00:39:03.877836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950389+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:39:03.950389+0000","last_peered":"2024-09-16T00:39:03.950389+0000","last_clean":"2024-09-16T00:39:03.950389+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:39:03.950389+0000","last_undegraded":"2024-09-16T00:39:03.950389+0000","last_fullsized":"2024-09-16T00:39:03.950389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.038107+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:39:04.038107+0000","last_peered":"2024-09-16T00:39:04.038107+0000","last_clean":"2024-09-16T00:39:04.038107+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:39:04.038107+0000","last_undegraded":"2024-09-16T00:39:04.038107+0000","last_fullsized":"2024-09-16T00:39:04.038107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:47.019916+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:39:47.019916+0000","last_peered":"2024-09-16T00:39:47.019916+0000","last_clean":"2024-09-16T00:39:47.019916+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:39:47.019916+0000","last_undegraded":"2024-09-16T00:39:47.019916+0000","last_fullsized":"2024-09-16T00:39:47.019916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037541+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:39:04.037541+0000","last_peered":"2024-09-16T00:39:04.037541+0000","last_clean":"2024-09-16T00:39:04.037541+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:39:04.037541+0000","last_undegraded":"2024-09-16T00:39:04.037541+0000","last_fullsized":"2024-09-16T00:39:04.037541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950097+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:39:03.950097+0000","last_peered":"2024-09-16T00:39:03.950097+0000","last_clean":"2024-09-16T00:39:03.950097+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:39:03.950097+0000","last_undegraded":"2024-09-16T00:39:03.950097+0000","last_fullsized":"2024-09-16T00:39:03.950097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951201+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:39:03.951201+0000","last_peered":"2024-09-16T00:39:03.951201+0000","last_clean":"2024-09-16T00:39:03.951201+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:39:03.951201+0000","last_undegraded":"2024-09-16T00:39:03.951201+0000","last_fullsized":"2024-09-16T00:39:03.951201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046477+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:39:03.046477+0000","last_peered":"2024-09-16T00:39:03.046477+0000","last_clean":"2024-09-16T00:39:03.046477+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:39:03.046477+0000","last_undegraded":"2024-09-16T00:39:03.046477+0000","last_fullsized":"2024-09-16T00:39:03.046477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950303+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:39:03.950303+0000","last_peered":"2024-09-16T00:39:03.950303+0000","last_clean":"2024-09-16T00:39:03.950303+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:39:03.950303+0000","last_undegraded":"2024-09-16T00:39:03.950303+0000","last_fullsized":"2024-09-16T00:39:03.950303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323726+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:39:03.323726+0000","last_peered":"2024-09-16T00:39:03.323726+0000","last_clean":"2024-09-16T00:39:03.323726+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:39:03.323726+0000","last_undegraded":"2024-09-16T00:39:03.323726+0000","last_fullsized":"2024-09-16T00:39:03.323726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950837+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:39:03.950837+0000","last_peered":"2024-09-16T00:39:03.950837+0000","last_clean":"2024-09-16T00:39:03.950837+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:39:03.950837+0000","last_undegraded":"2024-09-16T00:39:03.950837+0000","last_fullsized":"2024-09-16T00:39:03.950837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950288+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:39:03.950288+0000","last_peered":"2024-09-16T00:39:03.950288+0000","last_clean":"2024-09-16T00:39:03.950288+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:39:03.950288+0000","last_undegraded":"2024-09-16T00:39:03.950288+0000","last_fullsized":"2024-09-16T00:39:03.950288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046907+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:39:03.046907+0000","last_peered":"2024-09-16T00:39:03.046907+0000","last_clean":"2024-09-16T00:39:03.046907+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:39:03.046907+0000","last_undegraded":"2024-09-16T00:39:03.046907+0000","last_fullsized":"2024-09-16T00:39:03.046907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949991+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:39:03.949991+0000","last_peered":"2024-09-16T00:39:03.949991+0000","last_clean":"2024-09-16T00:39:03.949991+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:39:03.949991+0000","last_undegraded":"2024-09-16T00:39:03.949991+0000","last_fullsized":"2024-09-16T00:39:03.949991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950550+0000","last_change":"2024-09-16T00:38:52.972145+0000","last_active":"2024-09-16T00:39:03.950550+0000","last_peered":"2024-09-16T00:39:03.950550+0000","last_clean":"2024-09-16T00:39:03.950550+0000","last_became_active":"2024-09-16T00:38:52.971847+0000","last_became_peered":"2024-09-16T00:38:52.971847+0000","last_unstale":"2024-09-16T00:39:03.950550+0000","last_undegraded":"2024-09-16T00:39:03.950550+0000","last_fullsized":"2024-09-16T00:39:03.950550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:32.819389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949586+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:39:03.949586+0000","last_peered":"2024-09-16T00:39:03.949586+0000","last_clean":"2024-09-16T00:39:03.949586+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:39:03.949586+0000","last_undegraded":"2024-09-16T00:39:03.949586+0000","last_fullsized":"2024-09-16T00:39:03.949586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950106+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:39:03.950106+0000","last_peered":"2024-09-16T00:39:03.950106+0000","last_clean":"2024-09-16T00:39:03.950106+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:39:03.950106+0000","last_undegraded":"2024-09-16T00:39:03.950106+0000","last_fullsized":"2024-09-16T00:39:03.950106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949465+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:39:03.949465+0000","last_peered":"2024-09-16T00:39:03.949465+0000","last_clean":"2024-09-16T00:39:03.949465+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:39:03.949465+0000","last_undegraded":"2024-09-16T00:39:03.949465+0000","last_fullsized":"2024-09-16T00:39:03.949465+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046648+0000","last_change":"2024-09-16T00:38:50.875537+0000","last_active":"2024-09-16T00:39:03.046648+0000","last_peered":"2024-09-16T00:39:03.046648+0000","last_clean":"2024-09-16T00:39:03.046648+0000","last_became_active":"2024-09-16T00:38:50.875226+0000","last_became_peered":"2024-09-16T00:38:50.875226+0000","last_unstale":"2024-09-16T00:39:03.046648+0000","last_undegraded":"2024-09-16T00:39:03.046648+0000","last_fullsized":"2024-09-16T00:39:03.046648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:21.256432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951182+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:39:03.951182+0000","last_peered":"2024-09-16T00:39:03.951182+0000","last_clean":"2024-09-16T00:39:03.951182+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:39:03.951182+0000","last_undegraded":"2024-09-16T00:39:03.951182+0000","last_fullsized":"2024-09-16T00:39:03.951182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037837+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:39:04.037837+0000","last_peered":"2024-09-16T00:39:04.037837+0000","last_clean":"2024-09-16T00:39:04.037837+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:39:04.037837+0000","last_undegraded":"2024-09-16T00:39:04.037837+0000","last_fullsized":"2024-09-16T00:39:04.037837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877603+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:39:03.877603+0000","last_peered":"2024-09-16T00:39:03.877603+0000","last_clean":"2024-09-16T00:39:03.877603+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:39:03.877603+0000","last_undegraded":"2024-09-16T00:39:03.877603+0000","last_fullsized":"2024-09-16T00:39:03.877603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046796+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:39:03.046796+0000","last_peered":"2024-09-16T00:39:03.046796+0000","last_clean":"2024-09-16T00:39:03.046796+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:39:03.046796+0000","last_undegraded":"2024-09-16T00:39:03.046796+0000","last_fullsized":"2024-09-16T00:39:03.046796+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.527298+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:39:03.527298+0000","last_peered":"2024-09-16T00:39:03.527298+0000","last_clean":"2024-09-16T00:39:03.527298+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:39:03.527298+0000","last_undegraded":"2024-09-16T00:39:03.527298+0000","last_fullsized":"2024-09-16T00:39:03.527298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047393+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:39:03.047393+0000","last_peered":"2024-09-16T00:39:03.047393+0000","last_clean":"2024-09-16T00:39:03.047393+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:39:03.047393+0000","last_undegraded":"2024-09-16T00:39:03.047393+0000","last_fullsized":"2024-09-16T00:39:03.047393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950829+0000","last_change":"2024-09-16T00:38:51.423508+0000","last_active":"2024-09-16T00:39:03.950829+0000","last_peered":"2024-09-16T00:39:03.950829+0000","last_clean":"2024-09-16T00:39:03.950829+0000","last_became_active":"2024-09-16T00:38:51.423012+0000","last_became_peered":"2024-09-16T00:38:51.423012+0000","last_unstale":"2024-09-16T00:39:03.950829+0000","last_undegraded":"2024-09-16T00:39:03.950829+0000","last_fullsized":"2024-09-16T00:39:03.950829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:48.895796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576223+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:39:03.576223+0000","last_peered":"2024-09-16T00:39:03.576223+0000","last_clean":"2024-09-16T00:39:03.576223+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:39:03.576223+0000","last_undegraded":"2024-09-16T00:39:03.576223+0000","last_fullsized":"2024-09-16T00:39:03.576223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047184+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:39:03.047184+0000","last_peered":"2024-09-16T00:39:03.047184+0000","last_clean":"2024-09-16T00:39:03.047184+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:39:03.047184+0000","last_undegraded":"2024-09-16T00:39:03.047184+0000","last_fullsized":"2024-09-16T00:39:03.047184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.952197+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:39:03.952197+0000","last_peered":"2024-09-16T00:39:03.952197+0000","last_clean":"2024-09-16T00:39:03.952197+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:39:03.952197+0000","last_undegraded":"2024-09-16T00:39:03.952197+0000","last_fullsized":"2024-09-16T00:39:03.952197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045910+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:39:03.045910+0000","last_peered":"2024-09-16T00:39:03.045910+0000","last_clean":"2024-09-16T00:39:03.045910+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:39:03.045910+0000","last_undegraded":"2024-09-16T00:39:03.045910+0000","last_fullsized":"2024-09-16T00:39:03.045910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047257+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:39:03.047257+0000","last_peered":"2024-09-16T00:39:03.047257+0000","last_clean":"2024-09-16T00:39:03.047257+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:39:03.047257+0000","last_undegraded":"2024-09-16T00:39:03.047257+0000","last_fullsized":"2024-09-16T00:39:03.047257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950021+0000","last_change":"2024-09-16T00:38:50.875290+0000","last_active":"2024-09-16T00:39:03.950021+0000","last_peered":"2024-09-16T00:39:03.950021+0000","last_clean":"2024-09-16T00:39:03.950021+0000","last_became_active":"2024-09-16T00:38:50.874964+0000","last_became_peered":"2024-09-16T00:38:50.874964+0000","last_unstale":"2024-09-16T00:39:03.950021+0000","last_undegraded":"2024-09-16T00:39:03.950021+0000","last_fullsized":"2024-09-16T00:39:03.950021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:52.892725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950257+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:39:03.950257+0000","last_peered":"2024-09-16T00:39:03.950257+0000","last_clean":"2024-09-16T00:39:03.950257+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:39:03.950257+0000","last_undegraded":"2024-09-16T00:39:03.950257+0000","last_fullsized":"2024-09-16T00:39:03.950257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046776+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:39:03.046776+0000","last_peered":"2024-09-16T00:39:03.046776+0000","last_clean":"2024-09-16T00:39:03.046776+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:39:03.046776+0000","last_undegraded":"2024-09-16T00:39:03.046776+0000","last_fullsized":"2024-09-16T00:39:03.046776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951707+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:39:03.951707+0000","last_peered":"2024-09-16T00:39:03.951707+0000","last_clean":"2024-09-16T00:39:03.951707+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:39:03.951707+0000","last_undegraded":"2024-09-16T00:39:03.951707+0000","last_fullsized":"2024-09-16T00:39:03.951707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950428+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:39:03.950428+0000","last_peered":"2024-09-16T00:39:03.950428+0000","last_clean":"2024-09-16T00:39:03.950428+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:39:03.950428+0000","last_undegraded":"2024-09-16T00:39:03.950428+0000","last_fullsized":"2024-09-16T00:39:03.950428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950603+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:39:03.950603+0000","last_peered":"2024-09-16T00:39:03.950603+0000","last_clean":"2024-09-16T00:39:03.950603+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:39:03.950603+0000","last_undegraded":"2024-09-16T00:39:03.950603+0000","last_fullsized":"2024-09-16T00:39:03.950603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950466+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:39:03.950466+0000","last_peered":"2024-09-16T00:39:03.950466+0000","last_clean":"2024-09-16T00:39:03.950466+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:39:03.950466+0000","last_undegraded":"2024-09-16T00:39:03.950466+0000","last_fullsized":"2024-09-16T00:39:03.950466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950077+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:39:03.950077+0000","last_peered":"2024-09-16T00:39:03.950077+0000","last_clean":"2024-09-16T00:39:03.950077+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:39:03.950077+0000","last_undegraded":"2024-09-16T00:39:03.950077+0000","last_fullsized":"2024-09-16T00:39:03.950077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046570+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:39:03.046570+0000","last_peered":"2024-09-16T00:39:03.046570+0000","last_clean":"2024-09-16T00:39:03.046570+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:39:03.046570+0000","last_undegraded":"2024-09-16T00:39:03.046570+0000","last_fullsized":"2024-09-16T00:39:03.046570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877280+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:39:03.877280+0000","last_peered":"2024-09-16T00:39:03.877280+0000","last_clean":"2024-09-16T00:39:03.877280+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:39:03.877280+0000","last_undegraded":"2024-09-16T00:39:03.877280+0000","last_fullsized":"2024-09-16T00:39:03.877280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323969+0000","last_change":"2024-09-16T00:39:00.049744+0000","last_active":"2024-09-16T00:39:03.323969+0000","last_peered":"2024-09-16T00:39:03.323969+0000","last_clean":"2024-09-16T00:39:03.323969+0000","last_became_active":"2024-09-16T00:39:00.048919+0000","last_became_peered":"2024-09-16T00:39:00.048919+0000","last_unstale":"2024-09-16T00:39:03.323969+0000","last_undegraded":"2024-09-16T00:39:03.323969+0000","last_fullsized":"2024-09-16T00:39:03.323969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:49:19.509215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047002+0000","last_change":"2024-09-16T00:38:52.974330+0000","last_active":"2024-09-16T00:39:03.047002+0000","last_peered":"2024-09-16T00:39:03.047002+0000","last_clean":"2024-09-16T00:39:03.047002+0000","last_became_active":"2024-09-16T00:38:52.974004+0000","last_became_peered":"2024-09-16T00:38:52.974004+0000","last_unstale":"2024-09-16T00:39:03.047002+0000","last_undegraded":"2024-09-16T00:39:03.047002+0000","last_fullsized":"2024-09-16T00:39:03.047002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:31.820720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950818+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:39:03.950818+0000","last_peered":"2024-09-16T00:39:03.950818+0000","last_clean":"2024-09-16T00:39:03.950818+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:39:03.950818+0000","last_undegraded":"2024-09-16T00:39:03.950818+0000","last_fullsized":"2024-09-16T00:39:03.950818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046458+0000","last_change":"2024-09-16T00:39:00.053518+0000","last_active":"2024-09-16T00:39:03.046458+0000","last_peered":"2024-09-16T00:39:03.046458+0000","last_clean":"2024-09-16T00:39:03.046458+0000","last_became_active":"2024-09-16T00:39:00.052592+0000","last_became_peered":"2024-09-16T00:39:00.052592+0000","last_unstale":"2024-09-16T00:39:03.046458+0000","last_undegraded":"2024-09-16T00:39:03.046458+0000","last_fullsized":"2024-09-16T00:39:03.046458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:07:58.151992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949662+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:39:03.949662+0000","last_peered":"2024-09-16T00:39:03.949662+0000","last_clean":"2024-09-16T00:39:03.949662+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:39:03.949662+0000","last_undegraded":"2024-09-16T00:39:03.949662+0000","last_fullsized":"2024-09-16T00:39:03.949662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950664+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:39:03.950664+0000","last_peered":"2024-09-16T00:39:03.950664+0000","last_clean":"2024-09-16T00:39:03.950664+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:39:03.950664+0000","last_undegraded":"2024-09-16T00:39:03.950664+0000","last_fullsized":"2024-09-16T00:39:03.950664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951745+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:39:03.951745+0000","last_peered":"2024-09-16T00:39:03.951745+0000","last_clean":"2024-09-16T00:39:03.951745+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:39:03.951745+0000","last_undegraded":"2024-09-16T00:39:03.951745+0000","last_fullsized":"2024-09-16T00:39:03.951745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950700+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:39:03.950700+0000","last_peered":"2024-09-16T00:39:03.950700+0000","last_clean":"2024-09-16T00:39:03.950700+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:39:03.950700+0000","last_undegraded":"2024-09-16T00:39:03.950700+0000","last_fullsized":"2024-09-16T00:39:03.950700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'24","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:45.161142+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:39:45.161142+0000","last_peered":"2024-09-16T00:39:45.161142+0000","last_clean":"2024-09-16T00:39:45.161142+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:39:45.161142+0000","last_undegraded":"2024-09-16T00:39:45.161142+0000","last_fullsized":"2024-09-16T00:39:45.161142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":62,"num_write":24,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323462+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:39:03.323462+0000","last_peered":"2024-09-16T00:39:03.323462+0000","last_clean":"2024-09-16T00:39:03.323462+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:39:03.323462+0000","last_undegraded":"2024-09-16T00:39:03.323462+0000","last_fullsized":"2024-09-16T00:39:03.323462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576351+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:39:03.576351+0000","last_peered":"2024-09-16T00:39:03.576351+0000","last_clean":"2024-09-16T00:39:03.576351+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:39:03.576351+0000","last_undegraded":"2024-09-16T00:39:03.576351+0000","last_fullsized":"2024-09-16T00:39:03.576351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.952036+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:39:03.952036+0000","last_peered":"2024-09-16T00:39:03.952036+0000","last_clean":"2024-09-16T00:39:03.952036+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:39:03.952036+0000","last_undegraded":"2024-09-16T00:39:03.952036+0000","last_fullsized":"2024-09-16T00:39:03.952036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877436+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:39:03.877436+0000","last_peered":"2024-09-16T00:39:03.877436+0000","last_clean":"2024-09-16T00:39:03.877436+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:39:03.877436+0000","last_undegraded":"2024-09-16T00:39:03.877436+0000","last_fullsized":"2024-09-16T00:39:03.877436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046131+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:39:03.046131+0000","last_peered":"2024-09-16T00:39:03.046131+0000","last_clean":"2024-09-16T00:39:03.046131+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:39:03.046131+0000","last_undegraded":"2024-09-16T00:39:03.046131+0000","last_fullsized":"2024-09-16T00:39:03.046131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047184+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:39:03.047184+0000","last_peered":"2024-09-16T00:39:03.047184+0000","last_clean":"2024-09-16T00:39:03.047184+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:39:03.047184+0000","last_undegraded":"2024-09-16T00:39:03.047184+0000","last_fullsized":"2024-09-16T00:39:03.047184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323310+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:39:03.323310+0000","last_peered":"2024-09-16T00:39:03.323310+0000","last_clean":"2024-09-16T00:39:03.323310+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:39:03.323310+0000","last_undegraded":"2024-09-16T00:39:03.323310+0000","last_fullsized":"2024-09-16T00:39:03.323310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576575+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:39:03.576575+0000","last_peered":"2024-09-16T00:39:03.576575+0000","last_clean":"2024-09-16T00:39:03.576575+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:39:03.576575+0000","last_undegraded":"2024-09-16T00:39:03.576575+0000","last_fullsized":"2024-09-16T00:39:03.576575+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950559+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:39:03.950559+0000","last_peered":"2024-09-16T00:39:03.950559+0000","last_clean":"2024-09-16T00:39:03.950559+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:39:03.950559+0000","last_undegraded":"2024-09-16T00:39:03.950559+0000","last_fullsized":"2024-09-16T00:39:03.950559+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037706+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:39:04.037706+0000","last_peered":"2024-09-16T00:39:04.037706+0000","last_clean":"2024-09-16T00:39:04.037706+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:39:04.037706+0000","last_undegraded":"2024-09-16T00:39:04.037706+0000","last_fullsized":"2024-09-16T00:39:04.037706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045949+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:39:03.045949+0000","last_peered":"2024-09-16T00:39:03.045949+0000","last_clean":"2024-09-16T00:39:03.045949+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:39:03.045949+0000","last_undegraded":"2024-09-16T00:39:03.045949+0000","last_fullsized":"2024-09-16T00:39:03.045949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323357+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:39:03.323357+0000","last_peered":"2024-09-16T00:39:03.323357+0000","last_clean":"2024-09-16T00:39:03.323357+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:39:03.323357+0000","last_undegraded":"2024-09-16T00:39:03.323357+0000","last_fullsized":"2024-09-16T00:39:03.323357+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950717+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:39:03.950717+0000","last_peered":"2024-09-16T00:39:03.950717+0000","last_clean":"2024-09-16T00:39:03.950717+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:39:03.950717+0000","last_undegraded":"2024-09-16T00:39:03.950717+0000","last_fullsized":"2024-09-16T00:39:03.950717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046805+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:39:03.046805+0000","last_peered":"2024-09-16T00:39:03.046805+0000","last_clean":"2024-09-16T00:39:03.046805+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:39:03.046805+0000","last_undegraded":"2024-09-16T00:39:03.046805+0000","last_fullsized":"2024-09-16T00:39:03.046805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037643+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:39:04.037643+0000","last_peered":"2024-09-16T00:39:04.037643+0000","last_clean":"2024-09-16T00:39:04.037643+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:39:04.037643+0000","last_undegraded":"2024-09-16T00:39:04.037643+0000","last_fullsized":"2024-09-16T00:39:04.037643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949465+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:39:03.949465+0000","last_peered":"2024-09-16T00:39:03.949465+0000","last_clean":"2024-09-16T00:39:03.949465+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:39:03.949465+0000","last_undegraded":"2024-09-16T00:39:03.949465+0000","last_fullsized":"2024-09-16T00:39:03.949465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950221+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:39:03.950221+0000","last_peered":"2024-09-16T00:39:03.950221+0000","last_clean":"2024-09-16T00:39:03.950221+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:39:03.950221+0000","last_undegraded":"2024-09-16T00:39:03.950221+0000","last_fullsized":"2024-09-16T00:39:03.950221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951038+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:39:03.951038+0000","last_peered":"2024-09-16T00:39:03.951038+0000","last_clean":"2024-09-16T00:39:03.951038+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:39:03.951038+0000","last_undegraded":"2024-09-16T00:39:03.951038+0000","last_fullsized":"2024-09-16T00:39:03.951038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950457+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:39:03.950457+0000","last_peered":"2024-09-16T00:39:03.950457+0000","last_clean":"2024-09-16T00:39:03.950457+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:39:03.950457+0000","last_undegraded":"2024-09-16T00:39:03.950457+0000","last_fullsized":"2024-09-16T00:39:03.950457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951145+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:39:03.951145+0000","last_peered":"2024-09-16T00:39:03.951145+0000","last_clean":"2024-09-16T00:39:03.951145+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:39:03.951145+0000","last_undegraded":"2024-09-16T00:39:03.951145+0000","last_fullsized":"2024-09-16T00:39:03.951145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951025+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:39:03.951025+0000","last_peered":"2024-09-16T00:39:03.951025+0000","last_clean":"2024-09-16T00:39:03.951025+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:39:03.951025+0000","last_undegraded":"2024-09-16T00:39:03.951025+0000","last_fullsized":"2024-09-16T00:39:03.951025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951108+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:39:03.951108+0000","last_peered":"2024-09-16T00:39:03.951108+0000","last_clean":"2024-09-16T00:39:03.951108+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:39:03.951108+0000","last_undegraded":"2024-09-16T00:39:03.951108+0000","last_fullsized":"2024-09-16T00:39:03.951108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046274+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:39:03.046274+0000","last_peered":"2024-09-16T00:39:03.046274+0000","last_clean":"2024-09-16T00:39:03.046274+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:39:03.046274+0000","last_undegraded":"2024-09-16T00:39:03.046274+0000","last_fullsized":"2024-09-16T00:39:03.046274+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950637+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:39:03.950637+0000","last_peered":"2024-09-16T00:39:03.950637+0000","last_clean":"2024-09-16T00:39:03.950637+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:39:03.950637+0000","last_undegraded":"2024-09-16T00:39:03.950637+0000","last_fullsized":"2024-09-16T00:39:03.950637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950140+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:39:03.950140+0000","last_peered":"2024-09-16T00:39:03.950140+0000","last_clean":"2024-09-16T00:39:03.950140+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:39:03.950140+0000","last_undegraded":"2024-09-16T00:39:03.950140+0000","last_fullsized":"2024-09-16T00:39:03.950140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045819+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:39:03.045819+0000","last_peered":"2024-09-16T00:39:03.045819+0000","last_clean":"2024-09-16T00:39:03.045819+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:39:03.045819+0000","last_undegraded":"2024-09-16T00:39:03.045819+0000","last_fullsized":"2024-09-16T00:39:03.045819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950024+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:39:03.950024+0000","last_peered":"2024-09-16T00:39:03.950024+0000","last_clean":"2024-09-16T00:39:03.950024+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:39:03.950024+0000","last_undegraded":"2024-09-16T00:39:03.950024+0000","last_fullsized":"2024-09-16T00:39:03.950024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046730+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:39:03.046730+0000","last_peered":"2024-09-16T00:39:03.046730+0000","last_clean":"2024-09-16T00:39:03.046730+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:39:03.046730+0000","last_undegraded":"2024-09-16T00:39:03.046730+0000","last_fullsized":"2024-09-16T00:39:03.046730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950908+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:39:03.950908+0000","last_peered":"2024-09-16T00:39:03.950908+0000","last_clean":"2024-09-16T00:39:03.950908+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:39:03.950908+0000","last_undegraded":"2024-09-16T00:39:03.950908+0000","last_fullsized":"2024-09-16T00:39:03.950908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037971+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:39:04.037971+0000","last_peered":"2024-09-16T00:39:04.037971+0000","last_clean":"2024-09-16T00:39:04.037971+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:39:04.037971+0000","last_undegraded":"2024-09-16T00:39:04.037971+0000","last_fullsized":"2024-09-16T00:39:04.037971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950443+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:39:03.950443+0000","last_peered":"2024-09-16T00:39:03.950443+0000","last_clean":"2024-09-16T00:39:03.950443+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:39:03.950443+0000","last_undegraded":"2024-09-16T00:39:03.950443+0000","last_fullsized":"2024-09-16T00:39:03.950443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037733+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:39:04.037733+0000","last_peered":"2024-09-16T00:39:04.037733+0000","last_clean":"2024-09-16T00:39:04.037733+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:39:04.037733+0000","last_undegraded":"2024-09-16T00:39:04.037733+0000","last_fullsized":"2024-09-16T00:39:04.037733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576251+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:39:03.576251+0000","last_peered":"2024-09-16T00:39:03.576251+0000","last_clean":"2024-09-16T00:39:03.576251+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:39:03.576251+0000","last_undegraded":"2024-09-16T00:39:03.576251+0000","last_fullsized":"2024-09-16T00:39:03.576251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949846+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:39:03.949846+0000","last_peered":"2024-09-16T00:39:03.949846+0000","last_clean":"2024-09-16T00:39:03.949846+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:39:03.949846+0000","last_undegraded":"2024-09-16T00:39:03.949846+0000","last_fullsized":"2024-09-16T00:39:03.949846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046394+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:39:03.046394+0000","last_peered":"2024-09-16T00:39:03.046394+0000","last_clean":"2024-09-16T00:39:03.046394+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:39:03.046394+0000","last_undegraded":"2024-09-16T00:39:03.046394+0000","last_fullsized":"2024-09-16T00:39:03.046394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.951268+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:39:03.951268+0000","last_peered":"2024-09-16T00:39:03.951268+0000","last_clean":"2024-09-16T00:39:03.951268+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:39:03.951268+0000","last_undegraded":"2024-09-16T00:39:03.951268+0000","last_fullsized":"2024-09-16T00:39:03.951268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.323891+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:39:03.323891+0000","last_peered":"2024-09-16T00:39:03.323891+0000","last_clean":"2024-09-16T00:39:03.323891+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:39:03.323891+0000","last_undegraded":"2024-09-16T00:39:03.323891+0000","last_fullsized":"2024-09-16T00:39:03.323891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046967+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:39:03.046967+0000","last_peered":"2024-09-16T00:39:03.046967+0000","last_clean":"2024-09-16T00:39:03.046967+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:39:03.046967+0000","last_undegraded":"2024-09-16T00:39:03.046967+0000","last_fullsized":"2024-09-16T00:39:03.046967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.575931+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:39:03.575931+0000","last_peered":"2024-09-16T00:39:03.575931+0000","last_clean":"2024-09-16T00:39:03.575931+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:39:03.575931+0000","last_undegraded":"2024-09-16T00:39:03.575931+0000","last_fullsized":"2024-09-16T00:39:03.575931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045615+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:39:03.045615+0000","last_peered":"2024-09-16T00:39:03.045615+0000","last_clean":"2024-09-16T00:39:03.045615+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:39:03.045615+0000","last_undegraded":"2024-09-16T00:39:03.045615+0000","last_fullsized":"2024-09-16T00:39:03.045615+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045626+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:39:03.045626+0000","last_peered":"2024-09-16T00:39:03.045626+0000","last_clean":"2024-09-16T00:39:03.045626+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:39:03.045626+0000","last_undegraded":"2024-09-16T00:39:03.045626+0000","last_fullsized":"2024-09-16T00:39:03.045626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.949809+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:39:03.949809+0000","last_peered":"2024-09-16T00:39:03.949809+0000","last_clean":"2024-09-16T00:39:03.949809+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:39:03.949809+0000","last_undegraded":"2024-09-16T00:39:03.949809+0000","last_fullsized":"2024-09-16T00:39:03.949809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877612+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:39:03.877612+0000","last_peered":"2024-09-16T00:39:03.877612+0000","last_clean":"2024-09-16T00:39:03.877612+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:39:03.877612+0000","last_undegraded":"2024-09-16T00:39:03.877612+0000","last_fullsized":"2024-09-16T00:39:03.877612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.047387+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:39:03.047387+0000","last_peered":"2024-09-16T00:39:03.047387+0000","last_clean":"2024-09-16T00:39:03.047387+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:39:03.047387+0000","last_undegraded":"2024-09-16T00:39:03.047387+0000","last_fullsized":"2024-09-16T00:39:03.047387+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950638+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:39:03.950638+0000","last_peered":"2024-09-16T00:39:03.950638+0000","last_clean":"2024-09-16T00:39:03.950638+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:39:03.950638+0000","last_undegraded":"2024-09-16T00:39:03.950638+0000","last_fullsized":"2024-09-16T00:39:03.950638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.576567+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:39:03.576567+0000","last_peered":"2024-09-16T00:39:03.576567+0000","last_clean":"2024-09-16T00:39:03.576567+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:39:03.576567+0000","last_undegraded":"2024-09-16T00:39:03.576567+0000","last_fullsized":"2024-09-16T00:39:03.576567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:04.037546+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:39:04.037546+0000","last_peered":"2024-09-16T00:39:04.037546+0000","last_clean":"2024-09-16T00:39:04.037546+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:39:04.037546+0000","last_undegraded":"2024-09-16T00:39:04.037546+0000","last_fullsized":"2024-09-16T00:39:04.037546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046655+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:39:03.046655+0000","last_peered":"2024-09-16T00:39:03.046655+0000","last_clean":"2024-09-16T00:39:03.046655+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:39:03.046655+0000","last_undegraded":"2024-09-16T00:39:03.046655+0000","last_fullsized":"2024-09-16T00:39:03.046655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950213+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:39:03.950213+0000","last_peered":"2024-09-16T00:39:03.950213+0000","last_clean":"2024-09-16T00:39:03.950213+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:39:03.950213+0000","last_undegraded":"2024-09-16T00:39:03.950213+0000","last_fullsized":"2024-09-16T00:39:03.950213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.950858+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:39:03.950858+0000","last_peered":"2024-09-16T00:39:03.950858+0000","last_clean":"2024-09-16T00:39:03.950858+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:39:03.950858+0000","last_undegraded":"2024-09-16T00:39:03.950858+0000","last_fullsized":"2024-09-16T00:39:03.950858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.877571+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:39:03.877571+0000","last_peered":"2024-09-16T00:39:03.877571+0000","last_clean":"2024-09-16T00:39:03.877571+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:39:03.877571+0000","last_undegraded":"2024-09-16T00:39:03.877571+0000","last_fullsized":"2024-09-16T00:39:03.877571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045968+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:39:03.045968+0000","last_peered":"2024-09-16T00:39:03.045968+0000","last_clean":"2024-09-16T00:39:03.045968+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:39:03.045968+0000","last_undegraded":"2024-09-16T00:39:03.045968+0000","last_fullsized":"2024-09-16T00:39:03.045968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.045904+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:39:03.045904+0000","last_peered":"2024-09-16T00:39:03.045904+0000","last_clean":"2024-09-16T00:39:03.045904+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:39:03.045904+0000","last_undegraded":"2024-09-16T00:39:03.045904+0000","last_fullsized":"2024-09-16T00:39:03.045904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T00:39:03.046933+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:39:03.046933+0000","last_peered":"2024-09-16T00:39:03.046933+0000","last_clean":"2024-09-16T00:39:03.046933+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:39:03.046933+0000","last_undegraded":"2024-09-16T00:39:03.046933+0000","last_fullsized":"2024-09-16T00:39:03.046933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49896,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":35,"num_read_kb":68,"num_write":77,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":149688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547224,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":480695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1630208,"data_stored":1116259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939682,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":485373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":481299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1572864,"data_stored":1075741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":476599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1071245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":480717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":476703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":525731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":525697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":970752,"data_stored":476689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":49132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":49292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:39:52.339 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:39:52.339 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T00:39:52.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T00:39:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:53.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:52 smithi031 ceph-mon[30956]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:39:53.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:52 smithi031 ceph-mon[30956]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-09-16T00:39:53.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:52 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:53.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:52 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:53.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:52 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:39:53.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:52 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:39:53.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:52 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.099+0000 7fbf78738640 1 -- 172.21.15.31:0/102519944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf70107900 msgr2=0x7fbf70109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.099+0000 7fbf78738640 1 --2- 172.21.15.31:0/102519944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf70107900 0x7fbf70109d50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009ca0 tx=0x7fbf6802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.099+0000 7fbf78738640 1 -- 172.21.15.31:0/102519944 shutdown_connections 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.099+0000 7fbf78738640 1 --2- 172.21.15.31:0/102519944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf70107900 0x7fbf70109d50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.099+0000 7fbf78738640 1 --2- 172.21.15.31:0/102519944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf70104fd0 0x7fbf701073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.099+0000 7fbf78738640 1 --2- 172.21.15.31:0/102519944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf701026a0 0x7fbf70104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.099+0000 7fbf78738640 1 -- 172.21.15.31:0/102519944 >> 172.21.15.31:0/102519944 conn(0x7fbf700fc2a0 msgr2=0x7fbf700fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 -- 172.21.15.31:0/102519944 shutdown_connections 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 -- 172.21.15.31:0/102519944 wait complete. 2024-09-16T00:39:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 Processor -- start 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 -- start start 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf701026a0 0x7fbf70197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf70104fd0 0x7fbf70198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf70107900 0x7fbf7019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7010d8a0 con 0x7fbf701026a0 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7010d720 con 0x7fbf70104fd0 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf78738640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7010da20 con 0x7fbf70107900 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf764d4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf701026a0 0x7fbf70197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf764d4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf701026a0 0x7fbf70197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54226/0 (socket says 172.21.15.31:54226) 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.100+0000 7fbf764d4640 1 -- 172.21.15.31:0/1206738584 learned_addr learned my addr 172.21.15.31:0/1206738584 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf76cd5640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf70107900 0x7fbf7019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf764d4640 1 -- 172.21.15.31:0/1206738584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf70107900 msgr2=0x7fbf7019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf764d4640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf70107900 0x7fbf7019f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf764d4640 1 -- 172.21.15.31:0/1206738584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf70104fd0 msgr2=0x7fbf70198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf75cd3640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf70104fd0 0x7fbf70198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf764d4640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf70104fd0 0x7fbf70198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf764d4640 1 -- 172.21.15.31:0/1206738584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7019fb00 con 0x7fbf701026a0 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf76cd5640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf70107900 0x7fbf7019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf75cd3640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf70104fd0 0x7fbf70198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf764d4640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf701026a0 0x7fbf70197ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400d950 tx=0x7fbf6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf637fe640 1 -- 172.21.15.31:0/1206738584 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64017070 con 0x7fbf701026a0 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf637fe640 1 -- 172.21.15.31:0/1206738584 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf64009c30 con 0x7fbf701026a0 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7019fdf0 con 0x7fbf701026a0 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf637fe640 1 -- 172.21.15.31:0/1206738584 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400f6b0 con 0x7fbf701026a0 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.101+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf701a0240 con 0x7fbf701026a0 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.102+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf700fdd90 con 0x7fbf701026a0 2024-09-16T00:39:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.103+0000 7fbf637fe640 1 -- 172.21.15.31:0/1206738584 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbf640109d0 con 0x7fbf701026a0 2024-09-16T00:39:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.104+0000 7fbf637fe640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf5c0a3300 0x7fbf5c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.104+0000 7fbf75cd3640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf5c0a3300 0x7fbf5c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.104+0000 7fbf637fe640 1 -- 172.21.15.31:0/1206738584 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fbf640cfa30 con 0x7fbf701026a0 2024-09-16T00:39:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.105+0000 7fbf75cd3640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf5c0a3300 0x7fbf5c0a57c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf701993f0 tx=0x7fbf6c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.108+0000 7fbf637fe640 1 -- 172.21.15.31:0/1206738584 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf64009de0 con 0x7fbf701026a0 2024-09-16T00:39:53.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:52 smithi081 ceph-mon[35539]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:39:53.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:52 smithi081 ceph-mon[35539]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-09-16T00:39:53.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:52 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:53.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:52 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:53.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:52 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:39:53.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:52 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:39:53.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:52 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:53.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:52 smithi102 ceph-mon[35364]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:39:53.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:52 smithi102 ceph-mon[35364]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-09-16T00:39:53.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:52 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:53.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:52 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:53.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:52 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:39:53.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:52 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:39:53.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:52 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:39:53.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.350+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fbf7002da20 con 0x7fbf701026a0 2024-09-16T00:39:53.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.352+0000 7fbf637fe640 1 -- 172.21.15.31:0/1206738584 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fbf6409c350 con 0x7fbf701026a0 2024-09-16T00:39:53.353 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-16T00:39:53.353 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBZfudmgS7/FBAAucIp2ixRilANV94Xk3xC5w== 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf5c0a3300 msgr2=0x7fbf5c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf5c0a3300 0x7fbf5c0a57c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf701993f0 tx=0x7fbf6c005e50 comp rx=0 tx=0).stop 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf701026a0 msgr2=0x7fbf70197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf701026a0 0x7fbf70197ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400d950 tx=0x7fbf6400de10 comp rx=0 tx=0).stop 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 shutdown_connections 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf5c0a3300 0x7fbf5c0a57c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf70107900 0x7fbf7019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf70104fd0 0x7fbf70198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 --2- 172.21.15.31:0/1206738584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf701026a0 0x7fbf70197ed0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.355+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 >> 172.21.15.31:0/1206738584 conn(0x7fbf700fc2a0 msgr2=0x7fbf70102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.356+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 shutdown_connections 2024-09-16T00:39:53.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:53.356+0000 7fbf78738640 1 -- 172.21.15.31:0/1206738584 wait complete. 2024-09-16T00:39:54.032 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:53 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1206738584' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:39:54.032 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:53 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1206738584' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:39:54.064 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:39:54.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T00:39:54.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T00:39:54.104 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T00:39:54.104 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T00:39:54.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T00:39:54.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:53 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1206738584' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:39:54.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:53 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1206738584' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:39:54.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:53 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1206738584' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:39:54.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:53 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1206738584' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:39:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:54.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.891+0000 7f11269d8640 1 -- 172.21.15.31:0/219119677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112010c170 msgr2=0x7f112010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:54.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.891+0000 7f11269d8640 1 --2- 172.21.15.31:0/219119677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112010c170 0x7f112010e610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1118009ca0 tx=0x7f111802a950 comp rx=0 tx=0).stop 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.891+0000 7f11269d8640 1 -- 172.21.15.31:0/219119677 shutdown_connections 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.891+0000 7f11269d8640 1 --2- 172.21.15.31:0/219119677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112010c170 0x7f112010e610 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.891+0000 7f11269d8640 1 --2- 172.21.15.31:0/219119677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1120103660 0x7f112010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.891+0000 7f11269d8640 1 --2- 172.21.15.31:0/219119677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1120102d40 0x7f1120103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.891+0000 7f11269d8640 1 -- 172.21.15.31:0/219119677 >> 172.21.15.31:0/219119677 conn(0x7f11200fc7a0 msgr2=0x7f11200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.891+0000 7f11269d8640 1 -- 172.21.15.31:0/219119677 shutdown_connections 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.891+0000 7f11269d8640 1 -- 172.21.15.31:0/219119677 wait complete. 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f11269d8640 1 Processor -- start 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f11269d8640 1 -- start start 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f11269d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1120102d40 0x7f11201009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f11269d8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1120103660 0x7f1120100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f11269d8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112010c170 0x7f1120101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f11269d8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1120111f50 con 0x7f1120102d40 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f11269d8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1120111dd0 con 0x7f1120103660 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f11269d8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11201120d0 con 0x7f112010c170 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f111ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1120102d40 0x7f11201009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f111ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1120102d40 0x7f11201009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54248/0 (socket says 172.21.15.31:54248) 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f111ffff640 1 -- 172.21.15.31:0/2540541242 learned_addr learned my addr 172.21.15.31:0/2540541242 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f1117fff640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1120103660 0x7f1120100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.892+0000 7f1124f75640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112010c170 0x7f1120101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f111ffff640 1 -- 172.21.15.31:0/2540541242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112010c170 msgr2=0x7f1120101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f111ffff640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112010c170 0x7f1120101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f111ffff640 1 -- 172.21.15.31:0/2540541242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1120103660 msgr2=0x7f1120100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f111ffff640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1120103660 0x7f1120100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f111ffff640 1 -- 172.21.15.31:0/2540541242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11201a6830 con 0x7f1120102d40 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f111ffff640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1120102d40 0x7f11201009a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f111000c920 tx=0x7f111000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f111dffb640 1 -- 172.21.15.31:0/2540541242 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110007c70 con 0x7f1120102d40 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f111dffb640 1 -- 172.21.15.31:0/2540541242 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1110007e10 con 0x7f1120102d40 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f111dffb640 1 -- 172.21.15.31:0/2540541242 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110018890 con 0x7f1120102d40 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11201a6b20 con 0x7f1120102d40 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.893+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11201a7010 con 0x7f1120102d40 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.894+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1120103120 con 0x7f1120102d40 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.895+0000 7f111dffb640 1 -- 172.21.15.31:0/2540541242 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1110018a70 con 0x7f1120102d40 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.896+0000 7f111dffb640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f110c0a32d0 0x7f110c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.896+0000 7f1117fff640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f110c0a32d0 0x7f110c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.896+0000 7f111dffb640 1 -- 172.21.15.31:0/2540541242 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f11100cfde0 con 0x7f1120102d40 2024-09-16T00:39:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.897+0000 7f1117fff640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f110c0a32d0 0x7f110c0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1120102430 tx=0x7f110800b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:54.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:54.900+0000 7f111dffb640 1 -- 172.21.15.31:0/2540541242 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1110014030 con 0x7f1120102d40 2024-09-16T00:39:55.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.125+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f11201a7350 con 0x7f1120102d40 2024-09-16T00:39:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.130+0000 7f111dffb640 1 -- 172.21.15.31:0/2540541242 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264047 (secure 0 0 0) 0x7f111009c700 con 0x7f1120102d40 2024-09-16T00:39:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:55.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f110c0a32d0 msgr2=0x7f110c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:55.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f110c0a32d0 0x7f110c0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1120102430 tx=0x7f110800b2e0 comp rx=0 tx=0).stop 2024-09-16T00:39:55.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1120102d40 msgr2=0x7f11201009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:55.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1120102d40 0x7f11201009a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f111000c920 tx=0x7f111000cde0 comp rx=0 tx=0).stop 2024-09-16T00:39:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 shutdown_connections 2024-09-16T00:39:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f110c0a32d0 0x7f110c0a5790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112010c170 0x7f1120101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1120103660 0x7f1120100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 --2- 172.21.15.31:0/2540541242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1120102d40 0x7f11201009a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 >> 172.21.15.31:0/2540541242 conn(0x7f11200fc7a0 msgr2=0x7f11200fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 shutdown_connections 2024-09-16T00:39:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:55.133+0000 7f11269d8640 1 -- 172.21.15.31:0/2540541242 wait complete. 2024-09-16T00:39:55.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:55 smithi031 ceph-mon[30956]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-09-16T00:39:55.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:55 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2540541242' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:39:55.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:55 smithi081 ceph-mon[35539]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-09-16T00:39:55.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:55 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2540541242' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:39:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:55 smithi102 ceph-mon[35364]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-09-16T00:39:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:55 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2540541242' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:39:55.837 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":502885538},{"type":"v1","addr":"172.21.15.31:6801","nonce":502885538}]},"active_addr":"172.21.15.31:6801/502885538","active_change":"2024-09-16T00:31:57.569290+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3644001294}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2675788865}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2230492820}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3103400321}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1605712612}]}]} 2024-09-16T00:39:55.842 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T00:39:55.842 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T00:39:55.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:39:56.103 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:56.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.559+0000 7fb244731640 1 -- 172.21.15.31:0/1796748607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c100e20 msgr2=0x7fb23c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:56.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.559+0000 7fb244731640 1 --2- 172.21.15.31:0/1796748607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c100e20 0x7fb23c101200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb234009900 tx=0x7fb23402a480 comp rx=0 tx=0).stop 2024-09-16T00:39:56.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.560+0000 7fb244731640 1 -- 172.21.15.31:0/1796748607 shutdown_connections 2024-09-16T00:39:56.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.560+0000 7fb244731640 1 --2- 172.21.15.31:0/1796748607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb23c102ef0 0x7fb23c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:56.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.560+0000 7fb244731640 1 --2- 172.21.15.31:0/1796748607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23c102570 0x7fb23c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:56.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.560+0000 7fb244731640 1 --2- 172.21.15.31:0/1796748607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c100e20 0x7fb23c101200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:56.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.560+0000 7fb244731640 1 -- 172.21.15.31:0/1796748607 >> 172.21.15.31:0/1796748607 conn(0x7fb23c0fc7d0 msgr2=0x7fb23c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:56.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.560+0000 7fb244731640 1 -- 172.21.15.31:0/1796748607 shutdown_connections 2024-09-16T00:39:56.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.560+0000 7fb244731640 1 -- 172.21.15.31:0/1796748607 wait complete. 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.560+0000 7fb244731640 1 Processor -- start 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.560+0000 7fb244731640 1 -- start start 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb244731640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23c100e20 0x7fb23c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb244731640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c102570 0x7fb23c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb244731640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb23c102ef0 0x7fb23c10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb244731640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1120a0 con 0x7fb23c102570 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb244731640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c111f20 con 0x7fb23c102ef0 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb244731640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c112220 con 0x7fb23c100e20 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb241ccc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c102570 0x7fb23c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb241ccc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c102570 0x7fb23c10ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54274/0 (socket says 172.21.15.31:54274) 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb241ccc640 1 -- 172.21.15.31:0/3457489171 learned_addr learned my addr 172.21.15.31:0/3457489171 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:56.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb242cce640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb23c102ef0 0x7fb23c10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb2424cd640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23c100e20 0x7fb23c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb241ccc640 1 -- 172.21.15.31:0/3457489171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23c100e20 msgr2=0x7fb23c10e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb241ccc640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23c100e20 0x7fb23c10e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb241ccc640 1 -- 172.21.15.31:0/3457489171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb23c102ef0 msgr2=0x7fb23c10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb241ccc640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb23c102ef0 0x7fb23c10ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb241ccc640 1 -- 172.21.15.31:0/3457489171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c1a69b0 con 0x7fb23c102570 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb2424cd640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23c100e20 0x7fb23c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb241ccc640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c102570 0x7fb23c10ea10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb23000d770 tx=0x7fb23000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.561+0000 7fb23b7fe640 1 -- 172.21.15.31:0/3457489171 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230017070 con 0x7fb23c102570 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.562+0000 7fb23b7fe640 1 -- 172.21.15.31:0/3457489171 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb230005500 con 0x7fb23c102570 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.562+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c1a6ca0 con 0x7fb23c102570 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.562+0000 7fb23b7fe640 1 -- 172.21.15.31:0/3457489171 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230004e00 con 0x7fb23c102570 2024-09-16T00:39:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.562+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1a7190 con 0x7fb23c102570 2024-09-16T00:39:56.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.563+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c101200 con 0x7fb23c102570 2024-09-16T00:39:56.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.564+0000 7fb23b7fe640 1 -- 172.21.15.31:0/3457489171 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb230004fa0 con 0x7fb23c102570 2024-09-16T00:39:56.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.564+0000 7fb23b7fe640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb2240a3350 0x7fb2240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:56.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.564+0000 7fb2424cd640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb2240a3350 0x7fb2240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:56.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.565+0000 7fb23b7fe640 1 -- 172.21.15.31:0/3457489171 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fb2300cf6a0 con 0x7fb23c102570 2024-09-16T00:39:56.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.565+0000 7fb2424cd640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb2240a3350 0x7fb2240a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb234005ab0 tx=0x7fb234005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:56.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.569+0000 7fb23b7fe640 1 -- 172.21.15.31:0/3457489171 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb23009c020 con 0x7fb23c102570 2024-09-16T00:39:56.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.745+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb23c109090 con 0x7fb23c102570 2024-09-16T00:39:56.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.747+0000 7fb23b7fe640 1 -- 172.21.15.31:0/3457489171 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24150 (secure 0 0 0) 0x7fb23009b920 con 0x7fb23c102570 2024-09-16T00:39:56.748 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:56.748 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:39:02.716519+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:39:56.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb2240a3350 msgr2=0x7fb2240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:56.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb2240a3350 0x7fb2240a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb234005ab0 tx=0x7fb234005a20 comp rx=0 tx=0).stop 2024-09-16T00:39:56.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c102570 msgr2=0x7fb23c10ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:56.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c102570 0x7fb23c10ea10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb23000d770 tx=0x7fb23000dc30 comp rx=0 tx=0).stop 2024-09-16T00:39:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 shutdown_connections 2024-09-16T00:39:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb23c102ef0 0x7fb23c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb23c102570 0x7fb23c10ea10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb2240a3350 0x7fb2240a5810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 --2- 172.21.15.31:0/3457489171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23c100e20 0x7fb23c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 >> 172.21.15.31:0/3457489171 conn(0x7fb23c0fc7d0 msgr2=0x7fb23c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 shutdown_connections 2024-09-16T00:39:56.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:56.749+0000 7fb244731640 1 -- 172.21.15.31:0/3457489171 wait complete. 2024-09-16T00:39:57.430 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T00:39:57.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:39:57.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:57 smithi031 ceph-mon[30956]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-09-16T00:39:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:57 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3457489171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:57.694 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:57 smithi081 ceph-mon[35539]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-09-16T00:39:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:57 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3457489171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:57.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:57 smithi102 ceph-mon[35364]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-09-16T00:39:57.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:57 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3457489171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.163+0000 7f41448a2640 1 -- 172.21.15.31:0/2840321014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140103780 msgr2=0x7f4140107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.163+0000 7f41448a2640 1 --2- 172.21.15.31:0/2840321014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140103780 0x7f4140107bc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4134009930 tx=0x7f413402a420 comp rx=0 tx=0).stop 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.163+0000 7f41448a2640 1 -- 172.21.15.31:0/2840321014 shutdown_connections 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.163+0000 7f41448a2640 1 --2- 172.21.15.31:0/2840321014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4140108300 0x7f414010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.163+0000 7f41448a2640 1 --2- 172.21.15.31:0/2840321014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140103780 0x7f4140107bc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.163+0000 7f41448a2640 1 --2- 172.21.15.31:0/2840321014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4140102dd0 0x7f41401031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.163+0000 7f41448a2640 1 -- 172.21.15.31:0/2840321014 >> 172.21.15.31:0/2840321014 conn(0x7f41400fc7d0 msgr2=0x7f41400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.163+0000 7f41448a2640 1 -- 172.21.15.31:0/2840321014 shutdown_connections 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.163+0000 7f41448a2640 1 -- 172.21.15.31:0/2840321014 wait complete. 2024-09-16T00:39:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f41448a2640 1 Processor -- start 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f41448a2640 1 -- start start 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f41448a2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4140102dd0 0x7f41401985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f41448a2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4140103780 0x7f4140198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f41448a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140108300 0x7f414019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f41448a2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4140112070 con 0x7f4140108300 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f41448a2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4140111ef0 con 0x7f4140102dd0 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f41448a2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f41401121f0 con 0x7f4140103780 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140108300 0x7f414019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140108300 0x7f414019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43044/0 (socket says 172.21.15.31:43044) 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413ed9d640 1 -- 172.21.15.31:0/3407441617 learned_addr learned my addr 172.21.15.31:0/3407441617 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413e59c640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4140102dd0 0x7f41401985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413dd9b640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4140103780 0x7f4140198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413ed9d640 1 -- 172.21.15.31:0/3407441617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4140103780 msgr2=0x7f4140198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413ed9d640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4140103780 0x7f4140198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413ed9d640 1 -- 172.21.15.31:0/3407441617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4140102dd0 msgr2=0x7f41401985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413ed9d640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4140102dd0 0x7f41401985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.164+0000 7f413ed9d640 1 -- 172.21.15.31:0/3407441617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414019d5a0 con 0x7f4140108300 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.165+0000 7f413ed9d640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140108300 0x7f414019cec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f413800eb10 tx=0x7f413800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.165+0000 7f412f7fe640 1 -- 172.21.15.31:0/3407441617 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138004b20 con 0x7f4140108300 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.165+0000 7f412f7fe640 1 -- 172.21.15.31:0/3407441617 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4138004cc0 con 0x7f4140108300 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.165+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414019d890 con 0x7f4140108300 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.165+0000 7f412f7fe640 1 -- 172.21.15.31:0/3407441617 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138010760 con 0x7f4140108300 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.165+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41401a5170 con 0x7f4140108300 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.166+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41401031b0 con 0x7f4140108300 2024-09-16T00:39:58.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.167+0000 7f412f7fe640 1 -- 172.21.15.31:0/3407441617 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4138005000 con 0x7f4140108300 2024-09-16T00:39:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.168+0000 7f412f7fe640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41280a3350 0x7f41280a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.168+0000 7f412f7fe640 1 -- 172.21.15.31:0/3407441617 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f41380d0380 con 0x7f4140108300 2024-09-16T00:39:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.168+0000 7f413e59c640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41280a3350 0x7f41280a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.169+0000 7f413e59c640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41280a3350 0x7f41280a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4130009bd0 tx=0x7f4130005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.175+0000 7f412f7fe640 1 -- 172.21.15.31:0/3407441617 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41380541b0 con 0x7f4140108300 2024-09-16T00:39:58.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.353+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f414010a290 con 0x7f4140108300 2024-09-16T00:39:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.355+0000 7f412f7fe640 1 -- 172.21.15.31:0/3407441617 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24150 (secure 0 0 0) 0x7f413809cd00 con 0x7f4140108300 2024-09-16T00:39:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:39:58.357 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:39:02.716519+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:39:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.357+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41280a3350 msgr2=0x7f41280a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.357+0000 7f41448a2640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41280a3350 0x7f41280a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4130009bd0 tx=0x7f4130005f90 comp rx=0 tx=0).stop 2024-09-16T00:39:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.357+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140108300 msgr2=0x7f414019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.357+0000 7f41448a2640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140108300 0x7f414019cec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f413800eb10 tx=0x7f413800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.357+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 shutdown_connections 2024-09-16T00:39:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.357+0000 7f41448a2640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4140108300 0x7f414019cec0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.357+0000 7f41448a2640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41280a3350 0x7f41280a5810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.358+0000 7f41448a2640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4140103780 0x7f4140198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.358+0000 7f41448a2640 1 --2- 172.21.15.31:0/3407441617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4140102dd0 0x7f41401985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:58.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.358+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 >> 172.21.15.31:0/3407441617 conn(0x7f41400fc7d0 msgr2=0x7f4140105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:58.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.358+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 shutdown_connections 2024-09-16T00:39:58.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:39:58.358+0000 7f41448a2640 1 -- 172.21.15.31:0/3407441617 wait complete. 2024-09-16T00:39:58.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]: dispatch 2024-09-16T00:39:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-09-16T00:39:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-16T00:39:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-16T00:39:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-16T00:39:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-09-16T00:39:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-16T00:39:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-16T00:39:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:58 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3407441617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]: dispatch 2024-09-16T00:39:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-09-16T00:39:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-16T00:39:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-16T00:39:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-16T00:39:58.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-09-16T00:39:58.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-16T00:39:58.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-16T00:39:58.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:58 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3407441617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]: dispatch 2024-09-16T00:39:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-09-16T00:39:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-16T00:39:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-16T00:39:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-16T00:39:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-09-16T00:39:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-16T00:39:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-16T00:39:58.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:58 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3407441617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:39:59.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T00:39:59.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T00:39:59.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T00:39:59.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T00:39:59.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T00:39:59.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T00:39:59.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T00:39:59.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T00:39:59.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-16T00:39:59.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-16T00:39:59.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-16T00:39:59.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-16T00:39:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-16T00:39:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]': finished 2024-09-16T00:39:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-09-16T00:39:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-16T00:39:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-16T00:39:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-16T00:39:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-09-16T00:39:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-16T00:39:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-16T00:39:59.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:39:59 smithi081 ceph-mon[35539]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]': finished 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-16T00:39:59.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:39:59 smithi102 ceph-mon[35364]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:39:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:39:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-16T00:39:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]': finished 2024-09-16T00:39:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-09-16T00:39:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-16T00:39:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-16T00:39:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-16T00:39:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-09-16T00:39:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-16T00:39:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-16T00:39:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:39:59 smithi031 ceph-mon[30956]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:40:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 -- 172.21.15.31:0/115536228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd67c10e500 msgr2=0x7fd67c110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 --2- 172.21.15.31:0/115536228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd67c10e500 0x7fd67c110910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009ca0 tx=0x7fd66c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 -- 172.21.15.31:0/115536228 shutdown_connections 2024-09-16T00:40:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 --2- 172.21.15.31:0/115536228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd67c10e500 0x7fd67c110910 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 --2- 172.21.15.31:0/115536228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd67c101440 0x7fd67c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 --2- 172.21.15.31:0/115536228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c100b20 0x7fd67c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 -- 172.21.15.31:0/115536228 >> 172.21.15.31:0/115536228 conn(0x7fd67c0fc7d0 msgr2=0x7fd67c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 -- 172.21.15.31:0/115536228 shutdown_connections 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 -- 172.21.15.31:0/115536228 wait complete. 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.375+0000 7fd68423f640 1 Processor -- start 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd68423f640 1 -- start start 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd68423f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd67c100b20 0x7fd67c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd68423f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c101440 0x7fd67c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd68423f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd67c10e500 0x7fd67c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd68423f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c114230 con 0x7fd67c10e500 2024-09-16T00:40:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd68423f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c1140b0 con 0x7fd67c100b20 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd68423f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c1143b0 con 0x7fd67c101440 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6827dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd67c10e500 0x7fd67c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6827dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd67c10e500 0x7fd67c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43062/0 (socket says 172.21.15.31:43062) 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6827dc640 1 -- 172.21.15.31:0/2110312683 learned_addr learned my addr 172.21.15.31:0/2110312683 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6817da640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c101440 0x7fd67c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd681fdb640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd67c100b20 0x7fd67c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6827dc640 1 -- 172.21.15.31:0/2110312683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c101440 msgr2=0x7fd67c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6827dc640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c101440 0x7fd67c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6827dc640 1 -- 172.21.15.31:0/2110312683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd67c100b20 msgr2=0x7fd67c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6827dc640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd67c100b20 0x7fd67c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:00.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6827dc640 1 -- 172.21.15.31:0/2110312683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c19f6a0 con 0x7fd67c10e500 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.376+0000 7fd6827dc640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd67c10e500 0x7fd67c19efc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd66c031f80 tx=0x7fd66c031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.377+0000 7fd67affd640 1 -- 172.21.15.31:0/2110312683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c003b70 con 0x7fd67c10e500 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.377+0000 7fd67affd640 1 -- 172.21.15.31:0/2110312683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd66c003d10 con 0x7fd67c10e500 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.377+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c19f930 con 0x7fd67c10e500 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.377+0000 7fd67affd640 1 -- 172.21.15.31:0/2110312683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c004760 con 0x7fd67c10e500 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.377+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c1a7270 con 0x7fd67c10e500 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.378+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd67c10fa90 con 0x7fd67c10e500 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.379+0000 7fd67affd640 1 -- 172.21.15.31:0/2110312683 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd66c02f710 con 0x7fd67c10e500 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.380+0000 7fd67affd640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6640a3350 0x7fd6640a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.380+0000 7fd681fdb640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6640a3350 0x7fd6640a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.380+0000 7fd67affd640 1 -- 172.21.15.31:0/2110312683 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fd66c0f10c0 con 0x7fd67c10e500 2024-09-16T00:40:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.380+0000 7fd67affd640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] conn(0x7fd6640ae420 0x7fd6640b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.380+0000 7fd67affd640 1 -- 172.21.15.31:0/2110312683 --> [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6640b0f90 con 0x7fd6640ae420 2024-09-16T00:40:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.380+0000 7fd67affd640 1 -- 172.21.15.31:0/2110312683 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd66c0f1510 con 0x7fd67c10e500 2024-09-16T00:40:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.381+0000 7fd6817da640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] conn(0x7fd6640ae420 0x7fd6640b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.381+0000 7fd681fdb640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6640a3350 0x7fd6640a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd67400abe0 tx=0x7fd67400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.381+0000 7fd6817da640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] conn(0x7fd6640ae420 0x7fd6640b08c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.382+0000 7fd67affd640 1 -- 172.21.15.31:0/2110312683 <== osd.3 v2:172.21.15.31:6810/3958016219 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6640b0f90 con 0x7fd6640ae420 2024-09-16T00:40:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.395+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 --> [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd67c06a850 con 0x7fd6640ae420 2024-09-16T00:40:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd67affd640 1 -- 172.21.15.31:0/2110312683 <== osd.3 v2:172.21.15.31:6810/3958016219 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd67c06a850 con 0x7fd6640ae420 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] conn(0x7fd6640ae420 msgr2=0x7fd6640b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] conn(0x7fd6640ae420 0x7fd6640b08c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6640a3350 msgr2=0x7fd6640a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6640a3350 0x7fd6640a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd67400abe0 tx=0x7fd67400a2d0 comp rx=0 tx=0).stop 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd67c10e500 msgr2=0x7fd67c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 --2- 172.21.15.31:0/2110312683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd67c10e500 0x7fd67c19efc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd66c031f80 tx=0x7fd66c031a60 comp rx=0 tx=0).stop 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd6827dc640 1 -- 172.21.15.31:0/2110312683 reap_dead start 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 shutdown_connections 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 >> 172.21.15.31:0/2110312683 conn(0x7fd67c0fc7d0 msgr2=0x7fd67c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 shutdown_connections 2024-09-16T00:40:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:00.396+0000 7fd68423f640 1 -- 172.21.15.31:0/2110312683 wait complete. 2024-09-16T00:40:00.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:00 smithi031 ceph-mon[30956]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:40:00.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:00 smithi031 ceph-mon[30956]: overall HEALTH_OK 2024-09-16T00:40:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:00.673 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:00.675 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:00.675 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:00 smithi081 ceph-mon[35539]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:40:00.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:00 smithi081 ceph-mon[35539]: overall HEALTH_OK 2024-09-16T00:40:00.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:00 smithi102 ceph-mon[35364]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:40:00.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:00 smithi102 ceph-mon[35364]: overall HEALTH_OK 2024-09-16T00:40:01.082 INFO:teuthology.orchestra.run.smithi031.stdout:103079215171 2024-09-16T00:40:01.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T00:40:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:01.491 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:01 smithi031 ceph-mon[30956]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-16T00:40:01.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:01 smithi081 ceph-mon[35539]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-16T00:40:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:01 smithi102 ceph-mon[35364]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-16T00:40:03.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.992+0000 7fe52db7f640 1 -- 172.21.15.31:0/2102418244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe528101420 msgr2=0x7fe52810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.992+0000 7fe52db7f640 1 --2- 172.21.15.31:0/2102418244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe528101420 0x7fe52810d950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe520009cd0 tx=0x7fe52002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.992+0000 7fe52db7f640 1 -- 172.21.15.31:0/2102418244 shutdown_connections 2024-09-16T00:40:03.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.992+0000 7fe52db7f640 1 --2- 172.21.15.31:0/2102418244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe528101420 0x7fe52810d950 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.992+0000 7fe52db7f640 1 --2- 172.21.15.31:0/2102418244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe528100a80 0x7fe528100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.992+0000 7fe52db7f640 1 --2- 172.21.15.31:0/2102418244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe528106a80 0x7fe528106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.992+0000 7fe52db7f640 1 -- 172.21.15.31:0/2102418244 >> 172.21.15.31:0/2102418244 conn(0x7fe5280fc7a0 msgr2=0x7fe5280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.992+0000 7fe52db7f640 1 -- 172.21.15.31:0/2102418244 shutdown_connections 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.992+0000 7fe52db7f640 1 -- 172.21.15.31:0/2102418244 wait complete. 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe52db7f640 1 Processor -- start 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe52db7f640 1 -- start start 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe52db7f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe528100a80 0x7fe528197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe52db7f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe528101420 0x7fe528197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe52db7f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe528106a80 0x7fe52819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe52db7f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe528110e80 con 0x7fe528106a80 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe52db7f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe528110d00 con 0x7fe528100a80 2024-09-16T00:40:03.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe52db7f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe528111000 con 0x7fe528101420 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe527fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe528106a80 0x7fe52819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe527fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe528106a80 0x7fe52819bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43090/0 (socket says 172.21.15.31:43090) 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.993+0000 7fe527fff640 1 -- 172.21.15.31:0/2453547754 learned_addr learned my addr 172.21.15.31:0/2453547754 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe5277fe640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe528100a80 0x7fe528197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe527fff640 1 -- 172.21.15.31:0/2453547754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe528101420 msgr2=0x7fe528197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe527fff640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe528101420 0x7fe528197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe527fff640 1 -- 172.21.15.31:0/2453547754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe528100a80 msgr2=0x7fe528197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe527fff640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe528100a80 0x7fe528197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe527fff640 1 -- 172.21.15.31:0/2453547754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52819c2d0 con 0x7fe528106a80 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe5277fe640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe528100a80 0x7fe528197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe527fff640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe528106a80 0x7fe52819bbf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe520005cc0 tx=0x7fe520032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe524ff9640 1 -- 172.21.15.31:0/2453547754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520004040 con 0x7fe528106a80 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe524ff9640 1 -- 172.21.15.31:0/2453547754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5200046c0 con 0x7fe528106a80 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe524ff9640 1 -- 172.21.15.31:0/2453547754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520030e00 con 0x7fe528106a80 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52819c560 con 0x7fe528106a80 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.994+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5281a3ea0 con 0x7fe528106a80 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.995+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe528062980 con 0x7fe528106a80 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.996+0000 7fe524ff9640 1 -- 172.21.15.31:0/2453547754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe5200305a0 con 0x7fe528106a80 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.997+0000 7fe524ff9640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe5100a32d0 0x7fe5100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.997+0000 7fe5277fe640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe5100a32d0 0x7fe5100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.997+0000 7fe524ff9640 1 -- 172.21.15.31:0/2453547754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fe52002e030 con 0x7fe528106a80 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.997+0000 7fe524ff9640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] conn(0x7fe5100ae3a0 0x7fe5100b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.997+0000 7fe524ff9640 1 -- 172.21.15.31:0/2453547754 --> [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe52004fe50 con 0x7fe5100ae3a0 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.997+0000 7fe524ff9640 1 -- 172.21.15.31:0/2453547754 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe5200f1ab0 con 0x7fe528106a80 2024-09-16T00:40:03.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.998+0000 7fe5277fe640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe5100a32d0 0x7fe5100a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe5180058c0 tx=0x7fe518005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:03.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.998+0000 7fe526ffd640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] conn(0x7fe5100ae3a0 0x7fe5100b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.998+0000 7fe526ffd640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] conn(0x7fe5100ae3a0 0x7fe5100b0840 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:03.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:02.999+0000 7fe524ff9640 1 -- 172.21.15.31:0/2453547754 <== osd.7 v2:172.21.15.81:6816/269260316 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe52004fe50 con 0x7fe5100ae3a0 2024-09-16T00:40:03.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.013+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 --> [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe528062e00 con 0x7fe5100ae3a0 2024-09-16T00:40:03.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.013+0000 7fe524ff9640 1 -- 172.21.15.31:0/2453547754 <== osd.7 v2:172.21.15.81:6816/269260316 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe528062e00 con 0x7fe5100ae3a0 2024-09-16T00:40:03.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 >> [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] conn(0x7fe5100ae3a0 msgr2=0x7fe5100b0840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] conn(0x7fe5100ae3a0 0x7fe5100b0840 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe5100a32d0 msgr2=0x7fe5100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe5100a32d0 0x7fe5100a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe5180058c0 tx=0x7fe518005cb0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe528106a80 msgr2=0x7fe52819bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 --2- 172.21.15.31:0/2453547754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe528106a80 0x7fe52819bbf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe520005cc0 tx=0x7fe520032f40 comp rx=0 tx=0).stop 2024-09-16T00:40:03.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe527fff640 1 -- 172.21.15.31:0/2453547754 reap_dead start 2024-09-16T00:40:03.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 shutdown_connections 2024-09-16T00:40:03.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 >> 172.21.15.31:0/2453547754 conn(0x7fe5280fc7a0 msgr2=0x7fe5280fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:03.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 shutdown_connections 2024-09-16T00:40:03.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.014+0000 7fe52db7f640 1 -- 172.21.15.31:0/2453547754 wait complete. 2024-09-16T00:40:03.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.192+0000 7f2af415c640 1 -- 172.21.15.31:0/2910049666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aec101280 msgr2=0x7f2aec10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.192+0000 7f2af415c640 1 --2- 172.21.15.31:0/2910049666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aec101280 0x7f2aec10df40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009900 tx=0x7f2adc02a410 comp rx=0 tx=0).stop 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.192+0000 7f2af415c640 1 -- 172.21.15.31:0/2910049666 shutdown_connections 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.192+0000 7f2af415c640 1 --2- 172.21.15.31:0/2910049666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec10e480 0x7f2aec110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.192+0000 7f2af415c640 1 --2- 172.21.15.31:0/2910049666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aec101280 0x7f2aec10df40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.192+0000 7f2af415c640 1 --2- 172.21.15.31:0/2910049666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2aec1008d0 0x7f2aec100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.192+0000 7f2af415c640 1 -- 172.21.15.31:0/2910049666 >> 172.21.15.31:0/2910049666 conn(0x7f2aec0fc7a0 msgr2=0x7f2aec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 -- 172.21.15.31:0/2910049666 shutdown_connections 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 -- 172.21.15.31:0/2910049666 wait complete. 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 Processor -- start 2024-09-16T00:40:03.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 -- start start 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec1008d0 0x7f2aec19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2aec101280 0x7f2aec19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aec10e480 0x7f2aec19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1141b0 con 0x7f2aec10e480 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec114030 con 0x7f2aec1008d0 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.193+0000 7f2af415c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec114330 con 0x7f2aec101280 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af26f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aec10e480 0x7f2aec19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af26f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aec10e480 0x7f2aec19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43104/0 (socket says 172.21.15.31:43104) 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af26f9640 1 -- 172.21.15.31:0/2623132542 learned_addr learned my addr 172.21.15.31:0/2623132542 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:03.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af16f7640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2aec101280 0x7f2aec19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af1ef8640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec1008d0 0x7f2aec19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af26f9640 1 -- 172.21.15.31:0/2623132542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2aec101280 msgr2=0x7f2aec19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af26f9640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2aec101280 0x7f2aec19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af26f9640 1 -- 172.21.15.31:0/2623132542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec1008d0 msgr2=0x7f2aec19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af26f9640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec1008d0 0x7f2aec19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af26f9640 1 -- 172.21.15.31:0/2623132542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec19f680 con 0x7f2aec10e480 2024-09-16T00:40:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af26f9640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aec10e480 0x7f2aec19efa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009fd0 tx=0x7f2ae400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2ae2ffd640 1 -- 172.21.15.31:0/2623132542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4004d20 con 0x7f2aec10e480 2024-09-16T00:40:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2ae2ffd640 1 -- 172.21.15.31:0/2623132542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae4005020 con 0x7f2aec10e480 2024-09-16T00:40:03.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec19f970 con 0x7f2aec10e480 2024-09-16T00:40:03.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.194+0000 7f2ae2ffd640 1 -- 172.21.15.31:0/2623132542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40116d0 con 0x7f2aec10e480 2024-09-16T00:40:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.195+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec1a7250 con 0x7f2aec10e480 2024-09-16T00:40:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.195+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2aec062980 con 0x7f2aec10e480 2024-09-16T00:40:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.197+0000 7f2ae2ffd640 1 -- 172.21.15.31:0/2623132542 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2ae40051d0 con 0x7f2aec10e480 2024-09-16T00:40:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.197+0000 7f2ae2ffd640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2ad80a32d0 0x7f2ad80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.197+0000 7f2ae2ffd640 1 -- 172.21.15.31:0/2623132542 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f2ae40d0440 con 0x7f2aec10e480 2024-09-16T00:40:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.197+0000 7f2af1ef8640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2ad80a32d0 0x7f2ad80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.198+0000 7f2ae2ffd640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] conn(0x7f2ad80ae3a0 0x7f2ad80b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.198+0000 7f2ae2ffd640 1 -- 172.21.15.31:0/2623132542 --> [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ae4014ec0 con 0x7f2ad80ae3a0 2024-09-16T00:40:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.198+0000 7f2ae2ffd640 1 -- 172.21.15.31:0/2623132542 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f2ae40d0890 con 0x7f2aec10e480 2024-09-16T00:40:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.198+0000 7f2af16f7640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] conn(0x7f2ad80ae3a0 0x7f2ad80b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.198+0000 7f2af1ef8640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2ad80a32d0 0x7f2ad80a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009bd0 tx=0x7f2ae8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.198+0000 7f2af16f7640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] conn(0x7f2ad80ae3a0 0x7f2ad80b0840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.199+0000 7f2ae2ffd640 1 -- 172.21.15.31:0/2623132542 <== osd.9 v2:172.21.15.31:6826/3465344894 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ae4014ec0 con 0x7f2ad80ae3a0 2024-09-16T00:40:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.212+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 --> [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2aec062e00 con 0x7f2ad80ae3a0 2024-09-16T00:40:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2ae2ffd640 1 -- 172.21.15.31:0/2623132542 <== osd.9 v2:172.21.15.31:6826/3465344894 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2aec062e00 con 0x7f2ad80ae3a0 2024-09-16T00:40:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] conn(0x7f2ad80ae3a0 msgr2=0x7f2ad80b0840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] conn(0x7f2ad80ae3a0 0x7f2ad80b0840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2ad80a32d0 msgr2=0x7f2ad80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2ad80a32d0 0x7f2ad80a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009bd0 tx=0x7f2ae8005f90 comp rx=0 tx=0).stop 2024-09-16T00:40:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aec10e480 msgr2=0x7f2aec19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 --2- 172.21.15.31:0/2623132542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aec10e480 0x7f2aec19efa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009fd0 tx=0x7f2ae400ee60 comp rx=0 tx=0).stop 2024-09-16T00:40:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af26f9640 1 -- 172.21.15.31:0/2623132542 reap_dead start 2024-09-16T00:40:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 shutdown_connections 2024-09-16T00:40:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 >> 172.21.15.31:0/2623132542 conn(0x7f2aec0fc7a0 msgr2=0x7f2aec10f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 shutdown_connections 2024-09-16T00:40:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.213+0000 7f2af415c640 1 -- 172.21.15.31:0/2623132542 wait complete. 2024-09-16T00:40:03.682 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:03 smithi031 ceph-mon[30956]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-16T00:40:03.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:03 smithi081 ceph-mon[35539]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-16T00:40:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.766+0000 7fd4e10cb640 1 -- 172.21.15.31:0/243224881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc101df0 msgr2=0x7fd4dc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.766+0000 7fd4e10cb640 1 --2- 172.21.15.31:0/243224881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc101df0 0x7fd4dc1021d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009ca0 tx=0x7fd4cc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.766+0000 7fd4e10cb640 1 -- 172.21.15.31:0/243224881 shutdown_connections 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.766+0000 7fd4e10cb640 1 --2- 172.21.15.31:0/243224881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4dc106f20 0x7fd4dc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.766+0000 7fd4e10cb640 1 --2- 172.21.15.31:0/243224881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc1027a0 0x7fd4dc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.766+0000 7fd4e10cb640 1 --2- 172.21.15.31:0/243224881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc101df0 0x7fd4dc1021d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.766+0000 7fd4e10cb640 1 -- 172.21.15.31:0/243224881 >> 172.21.15.31:0/243224881 conn(0x7fd4dc0fd8f0 msgr2=0x7fd4dc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.767+0000 7fd4e10cb640 1 -- 172.21.15.31:0/243224881 shutdown_connections 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.767+0000 7fd4e10cb640 1 -- 172.21.15.31:0/243224881 wait complete. 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.767+0000 7fd4e10cb640 1 Processor -- start 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4e10cb640 1 -- start start 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4e10cb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc101df0 0x7fd4dc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4e10cb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc1027a0 0x7fd4dc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4e10cb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4dc106f20 0x7fd4dc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4e10cb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc111f80 con 0x7fd4dc1027a0 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4e10cb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc111e00 con 0x7fd4dc101df0 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4e10cb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc112100 con 0x7fd4dc106f20 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4da59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc1027a0 0x7fd4dc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4da59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc1027a0 0x7fd4dc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43130/0 (socket says 172.21.15.31:43130) 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4da59c640 1 -- 172.21.15.31:0/134524769 learned_addr learned my addr 172.21.15.31:0/134524769 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4dad9d640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc101df0 0x7fd4dc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4db59e640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4dc106f20 0x7fd4dc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4da59c640 1 -- 172.21.15.31:0/134524769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4dc106f20 msgr2=0x7fd4dc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4da59c640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4dc106f20 0x7fd4dc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4da59c640 1 -- 172.21.15.31:0/134524769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc101df0 msgr2=0x7fd4dc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4da59c640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc101df0 0x7fd4dc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4da59c640 1 -- 172.21.15.31:0/134524769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc1a6880 con 0x7fd4dc1027a0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.768+0000 7fd4dad9d640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4dc101df0 0x7fd4dc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.769+0000 7fd4da59c640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc1027a0 0x7fd4dc074a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c920 tx=0x7fd4c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.769+0000 7fd4d3fff640 1 -- 172.21.15.31:0/134524769 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4007c70 con 0x7fd4dc1027a0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.769+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4dc1a6b70 con 0x7fd4dc1027a0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.769+0000 7fd4d3fff640 1 -- 172.21.15.31:0/134524769 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4c4007e10 con 0x7fd4dc1027a0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.769+0000 7fd4d3fff640 1 -- 172.21.15.31:0/134524769 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4018890 con 0x7fd4dc1027a0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.769+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4dc1a70b0 con 0x7fd4dc1027a0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.770+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd4dc0628c0 con 0x7fd4dc1027a0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.771+0000 7fd4d3fff640 1 -- 172.21.15.31:0/134524769 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd4c4018a70 con 0x7fd4dc1027a0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.772+0000 7fd4d3fff640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd4c80a32d0 0x7fd4c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.772+0000 7fd4dad9d640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd4c80a32d0 0x7fd4c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.772+0000 7fd4d3fff640 1 -- 172.21.15.31:0/134524769 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fd4c40cfff0 con 0x7fd4dc1027a0 2024-09-16T00:40:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.773+0000 7fd4d3fff640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] conn(0x7fd4c80ae3a0 0x7fd4c80b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.773+0000 7fd4d3fff640 1 -- 172.21.15.31:0/134524769 --> [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd4c4012ec0 con 0x7fd4c80ae3a0 2024-09-16T00:40:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.773+0000 7fd4d3fff640 1 -- 172.21.15.31:0/134524769 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd4c40d25f0 con 0x7fd4dc1027a0 2024-09-16T00:40:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.773+0000 7fd4dad9d640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd4c80a32d0 0x7fd4c80a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc006010 tx=0x7fd4cc0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.773+0000 7fd4db59e640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] conn(0x7fd4c80ae3a0 0x7fd4c80b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.773+0000 7fd4db59e640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] conn(0x7fd4c80ae3a0 0x7fd4c80b0840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.774+0000 7fd4d3fff640 1 -- 172.21.15.31:0/134524769 <== osd.1 v2:172.21.15.81:6800/3628874088 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd4c4012ec0 con 0x7fd4c80ae3a0 2024-09-16T00:40:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.788+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 --> [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd4dc062cf0 con 0x7fd4c80ae3a0 2024-09-16T00:40:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.788+0000 7fd4d3fff640 1 -- 172.21.15.31:0/134524769 <== osd.1 v2:172.21.15.81:6800/3628874088 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd4dc062cf0 con 0x7fd4c80ae3a0 2024-09-16T00:40:03.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 >> [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] conn(0x7fd4c80ae3a0 msgr2=0x7fd4c80b0840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] conn(0x7fd4c80ae3a0 0x7fd4c80b0840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd4c80a32d0 msgr2=0x7fd4c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd4c80a32d0 0x7fd4c80a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc006010 tx=0x7fd4cc0093f0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc1027a0 msgr2=0x7fd4dc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:03.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 --2- 172.21.15.31:0/134524769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc1027a0 0x7fd4dc074a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c920 tx=0x7fd4c400cde0 comp rx=0 tx=0).stop 2024-09-16T00:40:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4db59e640 1 -- 172.21.15.31:0/134524769 reap_dead start 2024-09-16T00:40:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 shutdown_connections 2024-09-16T00:40:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 >> 172.21.15.31:0/134524769 conn(0x7fd4dc0fd8f0 msgr2=0x7fd4dc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 shutdown_connections 2024-09-16T00:40:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:03.789+0000 7fd4e10cb640 1 -- 172.21.15.31:0/134524769 wait complete. 2024-09-16T00:40:03.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:03 smithi102 ceph-mon[35364]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-16T00:40:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.051+0000 7f75bc917640 1 -- 172.21.15.31:0/754889488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b81036e0 msgr2=0x7f75b8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.051+0000 7f75bc917640 1 --2- 172.21.15.31:0/754889488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b81036e0 0x7f75b8107b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009930 tx=0x7f75ac02a420 comp rx=0 tx=0).stop 2024-09-16T00:40:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.051+0000 7f75bc917640 1 -- 172.21.15.31:0/754889488 shutdown_connections 2024-09-16T00:40:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.051+0000 7f75bc917640 1 --2- 172.21.15.31:0/754889488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75b81082a0 0x7f75b810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.051+0000 7f75bc917640 1 --2- 172.21.15.31:0/754889488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b81036e0 0x7f75b8107b90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.051+0000 7f75bc917640 1 --2- 172.21.15.31:0/754889488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b8102d30 0x7f75b8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.051+0000 7f75bc917640 1 -- 172.21.15.31:0/754889488 >> 172.21.15.31:0/754889488 conn(0x7f75b80fc7d0 msgr2=0x7f75b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.051+0000 7f75bc917640 1 -- 172.21.15.31:0/754889488 shutdown_connections 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.051+0000 7f75bc917640 1 -- 172.21.15.31:0/754889488 wait complete. 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75bc917640 1 Processor -- start 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75bc917640 1 -- start start 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75bc917640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b8102d30 0x7f75b8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75bc917640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75b81036e0 0x7f75b8198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75bc917640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b81082a0 0x7f75b819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75bc917640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81120a0 con 0x7f75b81082a0 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75bc917640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8111f20 con 0x7f75b81036e0 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75bc917640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112220 con 0x7f75b8102d30 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75b6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b81082a0 0x7f75b819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75b6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b81082a0 0x7f75b819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43136/0 (socket says 172.21.15.31:43136) 2024-09-16T00:40:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75b6d9d640 1 -- 172.21.15.31:0/1100899800 learned_addr learned my addr 172.21.15.31:0/1100899800 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75b659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b8102d30 0x7f75b8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75b5d9b640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75b81036e0 0x7f75b8198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75b6d9d640 1 -- 172.21.15.31:0/1100899800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b8102d30 msgr2=0x7f75b8198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75b6d9d640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b8102d30 0x7f75b8198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75b6d9d640 1 -- 172.21.15.31:0/1100899800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75b81036e0 msgr2=0x7f75b8198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.052+0000 7f75b6d9d640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75b81036e0 0x7f75b8198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.053+0000 7f75b6d9d640 1 -- 172.21.15.31:0/1100899800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b819d5d0 con 0x7f75b81082a0 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.053+0000 7f75b659c640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b8102d30 0x7f75b8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.053+0000 7f75b6d9d640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b81082a0 0x7f75b819cef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f75b000eb10 tx=0x7f75b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.053+0000 7f75a77fe640 1 -- 172.21.15.31:0/1100899800 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0004b20 con 0x7f75b81082a0 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.053+0000 7f75a77fe640 1 -- 172.21.15.31:0/1100899800 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b0004cc0 con 0x7f75b81082a0 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.053+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b819d8c0 con 0x7f75b81082a0 2024-09-16T00:40:04.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.053+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b81a51a0 con 0x7f75b81082a0 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.053+0000 7f75a77fe640 1 -- 172.21.15.31:0/1100899800 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0010760 con 0x7f75b81082a0 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.054+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7598001030 con 0x7f75b81082a0 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.056+0000 7f75a77fe640 1 -- 172.21.15.31:0/1100899800 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f75b0005000 con 0x7f75b81082a0 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.056+0000 7f75a77fe640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75a00a3350 0x7f75a00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.056+0000 7f75b659c640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75a00a3350 0x7f75a00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.056+0000 7f75a77fe640 1 -- 172.21.15.31:0/1100899800 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f75b00d0580 con 0x7f75b81082a0 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.057+0000 7f75a77fe640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] conn(0x7f75a00ae420 0x7f75a00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.057+0000 7f75a77fe640 1 -- 172.21.15.31:0/1100899800 --> [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75b0017e50 con 0x7f75a00ae420 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.057+0000 7f75a77fe640 1 -- 172.21.15.31:0/1100899800 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f75b00d09d0 con 0x7f75b81082a0 2024-09-16T00:40:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.057+0000 7f75b5d9b640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] conn(0x7f75a00ae420 0x7f75a00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:04.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.057+0000 7f75b659c640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75a00a3350 0x7f75a00a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009fc0 tx=0x7f75a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:04.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.057+0000 7f75b5d9b640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] conn(0x7f75a00ae420 0x7f75a00b08c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:04.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.058+0000 7f75a77fe640 1 -- 172.21.15.31:0/1100899800 <== osd.5 v2:172.21.15.102:6808/215828552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75b0017e50 con 0x7f75a00ae420 2024-09-16T00:40:04.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 --> [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7598002dd0 con 0x7f75a00ae420 2024-09-16T00:40:04.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75a77fe640 1 -- 172.21.15.31:0/1100899800 <== osd.5 v2:172.21.15.102:6808/215828552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7598002dd0 con 0x7f75a00ae420 2024-09-16T00:40:04.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 >> [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] conn(0x7f75a00ae420 msgr2=0x7f75a00b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:04.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] conn(0x7f75a00ae420 0x7f75a00b08c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:04.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75a00a3350 msgr2=0x7f75a00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:04.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f75a00a3350 0x7f75a00a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009fc0 tx=0x7f75a8005f90 comp rx=0 tx=0).stop 2024-09-16T00:40:04.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b81082a0 msgr2=0x7f75b819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:04.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 --2- 172.21.15.31:0/1100899800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b81082a0 0x7f75b819cef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f75b000eb10 tx=0x7f75b000c2d0 comp rx=0 tx=0).stop 2024-09-16T00:40:04.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75b6d9d640 1 -- 172.21.15.31:0/1100899800 reap_dead start 2024-09-16T00:40:04.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 shutdown_connections 2024-09-16T00:40:04.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 >> 172.21.15.31:0/1100899800 conn(0x7f75b80fc7d0 msgr2=0x7f75b8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:04.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 shutdown_connections 2024-09-16T00:40:04.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:04.073+0000 7f75bc917640 1 -- 172.21.15.31:0/1100899800 wait complete. 2024-09-16T00:40:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 -- 172.21.15.31:0/1236716470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608100a80 msgr2=0x7f3608100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 --2- 172.21.15.31:0/1236716470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608100a80 0x7f3608100ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009930 tx=0x7f35f002a420 comp rx=0 tx=0).stop 2024-09-16T00:40:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 -- 172.21.15.31:0/1236716470 shutdown_connections 2024-09-16T00:40:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 --2- 172.21.15.31:0/1236716470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608101420 0x7f360810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 --2- 172.21.15.31:0/1236716470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608100a80 0x7f3608100ee0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 --2- 172.21.15.31:0/1236716470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3608106a80 0x7f3608106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 -- 172.21.15.31:0/1236716470 >> 172.21.15.31:0/1236716470 conn(0x7f36080fc7a0 msgr2=0x7f36080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 -- 172.21.15.31:0/1236716470 shutdown_connections 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 -- 172.21.15.31:0/1236716470 wait complete. 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 Processor -- start 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.030+0000 7f360efeb640 1 -- start start 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360efeb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608100a80 0x7f36081974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360efeb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3608101420 0x7f3608197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360efeb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608106a80 0x7f360819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360efeb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3608110f20 con 0x7f3608106a80 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360efeb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3608110da0 con 0x7f3608100a80 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360efeb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f36081110a0 con 0x7f3608101420 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360cd87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608100a80 0x7f36081974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360d588640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608106a80 0x7f360819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360cd87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608100a80 0x7f36081974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:46356/0 (socket says 172.21.15.31:46356) 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360cd87640 1 -- 172.21.15.31:0/953857072 learned_addr learned my addr 172.21.15.31:0/953857072 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f3607fff640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3608101420 0x7f3608197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360d588640 1 -- 172.21.15.31:0/953857072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3608101420 msgr2=0x7f3608197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360d588640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3608101420 0x7f3608197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360d588640 1 -- 172.21.15.31:0/953857072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608100a80 msgr2=0x7f36081974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360d588640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608100a80 0x7f36081974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360d588640 1 -- 172.21.15.31:0/953857072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360819c4a0 con 0x7f3608106a80 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360cd87640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608100a80 0x7f36081974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.031+0000 7f360d588640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608106a80 0x7f360819bdc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b410 tx=0x7f35f800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.032+0000 7f3605ffb640 1 -- 172.21.15.31:0/953857072 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8015020 con 0x7f3608106a80 2024-09-16T00:40:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.032+0000 7f3605ffb640 1 -- 172.21.15.31:0/953857072 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f8011d10 con 0x7f3608106a80 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.032+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360819c790 con 0x7f3608106a80 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.032+0000 7f3605ffb640 1 -- 172.21.15.31:0/953857072 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f800f2e0 con 0x7f3608106a80 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.032+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36081a4070 con 0x7f3608106a80 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.033+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3608062980 con 0x7f3608106a80 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.034+0000 7f3605ffb640 1 -- 172.21.15.31:0/953857072 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f35f8018020 con 0x7f3608106a80 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.034+0000 7f3605ffb640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35f40a32d0 0x7f35f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.035+0000 7f360cd87640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35f40a32d0 0x7f35f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.035+0000 7f3605ffb640 1 -- 172.21.15.31:0/953857072 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f35f80cfa70 con 0x7f3608106a80 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.035+0000 7f3605ffb640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] conn(0x7f35f40ae3a0 0x7f35f40b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.035+0000 7f3605ffb640 1 -- 172.21.15.31:0/953857072 --> [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35f8004450 con 0x7f35f40ae3a0 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.035+0000 7f3605ffb640 1 -- 172.21.15.31:0/953857072 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f35f80cfec0 con 0x7f3608106a80 2024-09-16T00:40:05.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.035+0000 7f360cd87640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35f40a32d0 0x7f35f40a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f360000c530 tx=0x7f360000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.035+0000 7f3607fff640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] conn(0x7f35f40ae3a0 0x7f35f40b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.036+0000 7f3607fff640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] conn(0x7f35f40ae3a0 0x7f35f40b0840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.036+0000 7f3605ffb640 1 -- 172.21.15.31:0/953857072 <== osd.10 v2:172.21.15.81:6824/1666424173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35f8004450 con 0x7f35f40ae3a0 2024-09-16T00:40:05.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.051+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 --> [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3608062e00 con 0x7f35f40ae3a0 2024-09-16T00:40:05.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.051+0000 7f3605ffb640 1 -- 172.21.15.31:0/953857072 <== osd.10 v2:172.21.15.81:6824/1666424173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3608062e00 con 0x7f35f40ae3a0 2024-09-16T00:40:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 >> [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] conn(0x7f35f40ae3a0 msgr2=0x7f35f40b0840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] conn(0x7f35f40ae3a0 0x7f35f40b0840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35f40a32d0 msgr2=0x7f35f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35f40a32d0 0x7f35f40a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f360000c530 tx=0x7f360000b040 comp rx=0 tx=0).stop 2024-09-16T00:40:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608106a80 msgr2=0x7f360819bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 --2- 172.21.15.31:0/953857072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608106a80 0x7f360819bdc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b410 tx=0x7f35f800b8d0 comp rx=0 tx=0).stop 2024-09-16T00:40:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360d588640 1 -- 172.21.15.31:0/953857072 reap_dead start 2024-09-16T00:40:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 shutdown_connections 2024-09-16T00:40:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 >> 172.21.15.31:0/953857072 conn(0x7f36080fc7a0 msgr2=0x7f36080fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 shutdown_connections 2024-09-16T00:40:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.052+0000 7f360efeb640 1 -- 172.21.15.31:0/953857072 wait complete. 2024-09-16T00:40:05.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:05 smithi031 ceph-mon[30956]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T00:40:05.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:05 smithi081 ceph-mon[35539]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T00:40:05.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:05 smithi102 ceph-mon[35364]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T00:40:05.872 INFO:teuthology.orchestra.run.smithi031.stdout:210453397552 2024-09-16T00:40:05.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T00:40:06.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.235+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3910147139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f410c180 msgr2=0x7f63f410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:06.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.235+0000 7f63fa9e4640 1 --2- 172.21.15.31:0/3910147139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f410c180 0x7f63f410e600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009ca0 tx=0x7f63ec02a950 comp rx=0 tx=0).stop 2024-09-16T00:40:06.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.236+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3910147139 shutdown_connections 2024-09-16T00:40:06.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.236+0000 7f63fa9e4640 1 --2- 172.21.15.31:0/3910147139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f410c180 0x7f63f410e600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.236+0000 7f63fa9e4640 1 --2- 172.21.15.31:0/3910147139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63f41036c0 0x7f63f410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.236+0000 7f63fa9e4640 1 --2- 172.21.15.31:0/3910147139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f4102da0 0x7f63f4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.236+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3910147139 >> 172.21.15.31:0/3910147139 conn(0x7f63f40fc7a0 msgr2=0x7f63f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.236+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3910147139 shutdown_connections 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.236+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3910147139 wait complete. 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.236+0000 7f63fa9e4640 1 Processor -- start 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.236+0000 7f63fa9e4640 1 -- start start 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63fa9e4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63f4102da0 0x7f63f41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63fa9e4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f41036c0 0x7f63f41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63fa9e4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f410c180 0x7f63f419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63fa9e4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f63f4111f20 con 0x7f63f410c180 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63fa9e4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f63f4111da0 con 0x7f63f41036c0 2024-09-16T00:40:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63fa9e4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63f41120a0 con 0x7f63f4102da0 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f8f81640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f410c180 0x7f63f419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f8f81640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f410c180 0x7f63f419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43174/0 (socket says 172.21.15.31:43174) 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f8f81640 1 -- 172.21.15.31:0/3231630289 learned_addr learned my addr 172.21.15.31:0/3231630289 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63ebfff640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f41036c0 0x7f63f41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f3fff640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63f4102da0 0x7f63f41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f8f81640 1 -- 172.21.15.31:0/3231630289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63f4102da0 msgr2=0x7f63f41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f8f81640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63f4102da0 0x7f63f41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f8f81640 1 -- 172.21.15.31:0/3231630289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f41036c0 msgr2=0x7f63f41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f8f81640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f41036c0 0x7f63f41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f8f81640 1 -- 172.21.15.31:0/3231630289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f419d350 con 0x7f63f410c180 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63ebfff640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63f41036c0 0x7f63f41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.237+0000 7f63f8f81640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f410c180 0x7f63f419cc70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f63ec02ae70 tx=0x7f63ec02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.238+0000 7f63f1ffb640 1 -- 172.21.15.31:0/3231630289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec034040 con 0x7f63f410c180 2024-09-16T00:40:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.238+0000 7f63f1ffb640 1 -- 172.21.15.31:0/3231630289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ec030920 con 0x7f63f410c180 2024-09-16T00:40:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.238+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f419d5e0 con 0x7f63f410c180 2024-09-16T00:40:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.238+0000 7f63f1ffb640 1 -- 172.21.15.31:0/3231630289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec045480 con 0x7f63f410c180 2024-09-16T00:40:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.238+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f4100930 con 0x7f63f410c180 2024-09-16T00:40:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.239+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63f4062940 con 0x7f63f410c180 2024-09-16T00:40:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.240+0000 7f63f1ffb640 1 -- 172.21.15.31:0/3231630289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f63ec037020 con 0x7f63f410c180 2024-09-16T00:40:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.241+0000 7f63f1ffb640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f63e00a32d0 0x7f63e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.241+0000 7f63f3fff640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f63e00a32d0 0x7f63e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.241+0000 7f63f1ffb640 1 -- 172.21.15.31:0/3231630289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f63ec0f0f70 con 0x7f63f410c180 2024-09-16T00:40:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.241+0000 7f63f1ffb640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] conn(0x7f63e00ae3a0 0x7f63e00b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.241+0000 7f63f1ffb640 1 -- 172.21.15.31:0/3231630289 --> [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63ec02c030 con 0x7f63e00ae3a0 2024-09-16T00:40:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.241+0000 7f63f1ffb640 1 -- 172.21.15.31:0/3231630289 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f63ec0f1320 con 0x7f63f410c180 2024-09-16T00:40:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.241+0000 7f63f3fff640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f63e00a32d0 0x7f63e00a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63e40058c0 tx=0x7f63e4010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.241+0000 7f63ebfff640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] conn(0x7f63e00ae3a0 0x7f63e00b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.242+0000 7f63ebfff640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] conn(0x7f63e00ae3a0 0x7f63e00b0840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.242+0000 7f63f1ffb640 1 -- 172.21.15.31:0/3231630289 <== osd.8 v2:172.21.15.102:6816/304252744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63ec02c030 con 0x7f63e00ae3a0 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.256+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 --> [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f63f4062dc0 con 0x7f63e00ae3a0 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63f1ffb640 1 -- 172.21.15.31:0/3231630289 <== osd.8 v2:172.21.15.102:6816/304252744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f63f4062dc0 con 0x7f63e00ae3a0 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 >> [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] conn(0x7f63e00ae3a0 msgr2=0x7f63e00b0840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] conn(0x7f63e00ae3a0 0x7f63e00b0840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f63e00a32d0 msgr2=0x7f63e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f63e00a32d0 0x7f63e00a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63e40058c0 tx=0x7f63e4010040 comp rx=0 tx=0).stop 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f410c180 msgr2=0x7f63f419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 --2- 172.21.15.31:0/3231630289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63f410c180 0x7f63f419cc70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f63ec02ae70 tx=0x7f63ec02f5b0 comp rx=0 tx=0).stop 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63f8f81640 1 -- 172.21.15.31:0/3231630289 reap_dead start 2024-09-16T00:40:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 shutdown_connections 2024-09-16T00:40:06.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 >> 172.21.15.31:0/3231630289 conn(0x7f63f40fc7a0 msgr2=0x7f63f40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:06.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 shutdown_connections 2024-09-16T00:40:06.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:05.257+0000 7f63fa9e4640 1 -- 172.21.15.31:0/3231630289 wait complete. 2024-09-16T00:40:06.786 INFO:teuthology.orchestra.run.smithi031.stdout:158913790010 2024-09-16T00:40:06.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T00:40:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:07 smithi031 ceph-mon[30956]: pgmap v323: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T00:40:07.786 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:07 smithi081 ceph-mon[35539]: pgmap v323: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T00:40:07.786 INFO:teuthology.orchestra.run.smithi031.stdout:292057776161 2024-09-16T00:40:07.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-16T00:40:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:07 smithi102 ceph-mon[35364]: pgmap v323: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T00:40:07.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.858+0000 7fd6c19be640 1 -- 172.21.15.31:0/188359703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6bc106f20 msgr2=0x7fd6bc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.858+0000 7fd6c19be640 1 --2- 172.21.15.31:0/188359703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6bc106f20 0x7fd6bc10eaa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009ca0 tx=0x7fd6b402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.858+0000 7fd6c19be640 1 -- 172.21.15.31:0/188359703 shutdown_connections 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.858+0000 7fd6c19be640 1 --2- 172.21.15.31:0/188359703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6bc106f20 0x7fd6bc10eaa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.858+0000 7fd6c19be640 1 --2- 172.21.15.31:0/188359703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6bc1027a0 0x7fd6bc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.858+0000 7fd6c19be640 1 --2- 172.21.15.31:0/188359703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6bc101df0 0x7fd6bc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.858+0000 7fd6c19be640 1 -- 172.21.15.31:0/188359703 >> 172.21.15.31:0/188359703 conn(0x7fd6bc0fd8f0 msgr2=0x7fd6bc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.858+0000 7fd6c19be640 1 -- 172.21.15.31:0/188359703 shutdown_connections 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.858+0000 7fd6c19be640 1 -- 172.21.15.31:0/188359703 wait complete. 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.859+0000 7fd6c19be640 1 Processor -- start 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.859+0000 7fd6c19be640 1 -- start start 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.859+0000 7fd6c19be640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6bc101df0 0x7fd6bc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.859+0000 7fd6c19be640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6bc1027a0 0x7fd6bc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.859+0000 7fd6c19be640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6bc106f20 0x7fd6bc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.859+0000 7fd6c19be640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc111fa0 con 0x7fd6bc1027a0 2024-09-16T00:40:07.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.859+0000 7fd6c19be640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc111e20 con 0x7fd6bc106f20 2024-09-16T00:40:07.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.859+0000 7fd6c19be640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc112120 con 0x7fd6bc101df0 2024-09-16T00:40:07.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6ba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6bc1027a0 0x7fd6bc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6ba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6bc1027a0 0x7fd6bc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50492/0 (socket says 172.21.15.31:50492) 2024-09-16T00:40:07.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6ba7fc640 1 -- 172.21.15.31:0/1744146594 learned_addr learned my addr 172.21.15.31:0/1744146594 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6bb7fe640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6bc106f20 0x7fd6bc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6baffd640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6bc101df0 0x7fd6bc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6ba7fc640 1 -- 172.21.15.31:0/1744146594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6bc101df0 msgr2=0x7fd6bc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6ba7fc640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6bc101df0 0x7fd6bc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6ba7fc640 1 -- 172.21.15.31:0/1744146594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6bc106f20 msgr2=0x7fd6bc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6ba7fc640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6bc106f20 0x7fd6bc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6ba7fc640 1 -- 172.21.15.31:0/1744146594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc1a6920 con 0x7fd6bc1027a0 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6ba7fc640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6bc1027a0 0x7fd6bc074a30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b4a0 tx=0x7fd6b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6c09bc640 1 -- 172.21.15.31:0/1744146594 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0014070 con 0x7fd6bc1027a0 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc1a6c10 con 0x7fd6bc1027a0 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6c09bc640 1 -- 172.21.15.31:0/1744146594 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6b0010af0 con 0x7fd6bc1027a0 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.860+0000 7fd6c09bc640 1 -- 172.21.15.31:0/1744146594 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0017d10 con 0x7fd6bc1027a0 2024-09-16T00:40:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.861+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc1a71a0 con 0x7fd6bc1027a0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.861+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6bc062880 con 0x7fd6bc1027a0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.863+0000 7fd6c09bc640 1 -- 172.21.15.31:0/1744146594 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd6b0017070 con 0x7fd6bc1027a0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.864+0000 7fd6c09bc640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6a40a32d0 0x7fd6a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.864+0000 7fd6baffd640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6a40a32d0 0x7fd6a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.864+0000 7fd6c09bc640 1 -- 172.21.15.31:0/1744146594 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fd6b00cf2d0 con 0x7fd6bc1027a0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.864+0000 7fd6c09bc640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] conn(0x7fd6a40ae3a0 0x7fd6a40b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.864+0000 7fd6c09bc640 1 -- 172.21.15.31:0/1744146594 --> [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6b000dec0 con 0x7fd6a40ae3a0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.864+0000 7fd6c09bc640 1 -- 172.21.15.31:0/1744146594 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd6b00cf7b0 con 0x7fd6bc1027a0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.864+0000 7fd6bb7fe640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] conn(0x7fd6a40ae3a0 0x7fd6a40b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.864+0000 7fd6baffd640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6a40a32d0 0x7fd6a40a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac005d00 tx=0x7fd6ac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.864+0000 7fd6bb7fe640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] conn(0x7fd6a40ae3a0 0x7fd6a40b0840 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.865+0000 7fd6c09bc640 1 -- 172.21.15.31:0/1744146594 <== osd.6 v2:172.21.15.31:6818/1424825752 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6b000dec0 con 0x7fd6a40ae3a0 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.882+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 --> [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd6bc062cc0 con 0x7fd6a40ae3a0 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.882+0000 7fd6c09bc640 1 -- 172.21.15.31:0/1744146594 <== osd.6 v2:172.21.15.31:6818/1424825752 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd6bc062cc0 con 0x7fd6a40ae3a0 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] conn(0x7fd6a40ae3a0 msgr2=0x7fd6a40b0840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] conn(0x7fd6a40ae3a0 0x7fd6a40b0840 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6a40a32d0 msgr2=0x7fd6a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6a40a32d0 0x7fd6a40a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac005d00 tx=0x7fd6ac00b410 comp rx=0 tx=0).stop 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6bc1027a0 msgr2=0x7fd6bc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 --2- 172.21.15.31:0/1744146594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6bc1027a0 0x7fd6bc074a30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b4a0 tx=0x7fd6b000b960 comp rx=0 tx=0).stop 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6bb7fe640 1 -- 172.21.15.31:0/1744146594 reap_dead start 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 shutdown_connections 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 >> 172.21.15.31:0/1744146594 conn(0x7fd6bc0fd8f0 msgr2=0x7fd6bc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 shutdown_connections 2024-09-16T00:40:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:07.883+0000 7fd6c19be640 1 -- 172.21.15.31:0/1744146594 wait complete. 2024-09-16T00:40:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.711+0000 7feb65a90640 1 -- 172.21.15.31:0/2220889074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60100e20 msgr2=0x7feb60101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.711+0000 7feb65a90640 1 --2- 172.21.15.31:0/2220889074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60100e20 0x7feb60101200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feb50009900 tx=0x7feb5002a480 comp rx=0 tx=0).stop 2024-09-16T00:40:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.711+0000 7feb65a90640 1 -- 172.21.15.31:0/2220889074 shutdown_connections 2024-09-16T00:40:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.711+0000 7feb65a90640 1 --2- 172.21.15.31:0/2220889074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb60102ef0 0x7feb6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.711+0000 7feb65a90640 1 --2- 172.21.15.31:0/2220889074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb60102570 0x7feb601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.711+0000 7feb65a90640 1 --2- 172.21.15.31:0/2220889074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60100e20 0x7feb60101200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.711+0000 7feb65a90640 1 -- 172.21.15.31:0/2220889074 >> 172.21.15.31:0/2220889074 conn(0x7feb600fc7d0 msgr2=0x7feb600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.711+0000 7feb65a90640 1 -- 172.21.15.31:0/2220889074 shutdown_connections 2024-09-16T00:40:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.711+0000 7feb65a90640 1 -- 172.21.15.31:0/2220889074 wait complete. 2024-09-16T00:40:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.712+0000 7feb65a90640 1 Processor -- start 2024-09-16T00:40:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.712+0000 7feb65a90640 1 -- start start 2024-09-16T00:40:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.712+0000 7feb65a90640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb60100e20 0x7feb60199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.712+0000 7feb65a90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb60102570 0x7feb60199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.712+0000 7feb65a90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60102ef0 0x7feb601a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.712+0000 7feb65a90640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb60110f80 con 0x7feb60102ef0 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.712+0000 7feb65a90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7feb60110e00 con 0x7feb60102570 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.712+0000 7feb65a90640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feb60111100 con 0x7feb60100e20 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60102ef0 0x7feb601a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60102ef0 0x7feb601a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50510/0 (socket says 172.21.15.31:50510) 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5ffff640 1 -- 172.21.15.31:0/1638843070 learned_addr learned my addr 172.21.15.31:0/1638843070 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5effd640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb60102570 0x7feb60199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5f7fe640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb60100e20 0x7feb60199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5ffff640 1 -- 172.21.15.31:0/1638843070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb60100e20 msgr2=0x7feb60199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5ffff640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb60100e20 0x7feb60199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5ffff640 1 -- 172.21.15.31:0/1638843070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb60102570 msgr2=0x7feb60199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5ffff640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb60102570 0x7feb60199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5ffff640 1 -- 172.21.15.31:0/1638843070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb601a48f0 con 0x7feb60102ef0 2024-09-16T00:40:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5ffff640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60102ef0 0x7feb601a43b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feb5800eaa0 tx=0x7feb5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5cff9640 1 -- 172.21.15.31:0/1638843070 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5800cdb0 con 0x7feb60102ef0 2024-09-16T00:40:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5cff9640 1 -- 172.21.15.31:0/1638843070 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb58005020 con 0x7feb60102ef0 2024-09-16T00:40:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb601a4ac0 con 0x7feb60102ef0 2024-09-16T00:40:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb5cff9640 1 -- 172.21.15.31:0/1638843070 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb580116b0 con 0x7feb60102ef0 2024-09-16T00:40:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.713+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb601a4fd0 con 0x7feb60102ef0 2024-09-16T00:40:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.714+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb60062980 con 0x7feb60102ef0 2024-09-16T00:40:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.716+0000 7feb5cff9640 1 -- 172.21.15.31:0/1638843070 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7feb58005560 con 0x7feb60102ef0 2024-09-16T00:40:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.716+0000 7feb5cff9640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7feb480a32d0 0x7feb480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.716+0000 7feb5f7fe640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7feb480a32d0 0x7feb480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.716+0000 7feb5cff9640 1 -- 172.21.15.31:0/1638843070 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7feb580d03c0 con 0x7feb60102ef0 2024-09-16T00:40:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.717+0000 7feb5cff9640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] conn(0x7feb480ae3a0 0x7feb480b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.717+0000 7feb5cff9640 1 -- 172.21.15.31:0/1638843070 --> [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb58014ec0 con 0x7feb480ae3a0 2024-09-16T00:40:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.717+0000 7feb5cff9640 1 -- 172.21.15.31:0/1638843070 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7feb580d0770 con 0x7feb60102ef0 2024-09-16T00:40:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.717+0000 7feb5f7fe640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7feb480a32d0 0x7feb480a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb50005ab0 tx=0x7feb50005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.717+0000 7feb5effd640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] conn(0x7feb480ae3a0 0x7feb480b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.717+0000 7feb5effd640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] conn(0x7feb480ae3a0 0x7feb480b0840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.718+0000 7feb5cff9640 1 -- 172.21.15.31:0/1638843070 <== osd.2 v2:172.21.15.102:6800/2755894572 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb58014ec0 con 0x7feb480ae3a0 2024-09-16T00:40:08.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.732+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 --> [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb60062e00 con 0x7feb480ae3a0 2024-09-16T00:40:08.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.732+0000 7feb5cff9640 1 -- 172.21.15.31:0/1638843070 <== osd.2 v2:172.21.15.102:6800/2755894572 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feb60062e00 con 0x7feb480ae3a0 2024-09-16T00:40:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 >> [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] conn(0x7feb480ae3a0 msgr2=0x7feb480b0840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] conn(0x7feb480ae3a0 0x7feb480b0840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7feb480a32d0 msgr2=0x7feb480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7feb480a32d0 0x7feb480a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb50005ab0 tx=0x7feb50005a20 comp rx=0 tx=0).stop 2024-09-16T00:40:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60102ef0 msgr2=0x7feb601a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 --2- 172.21.15.31:0/1638843070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb60102ef0 0x7feb601a43b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feb5800eaa0 tx=0x7feb5800ef60 comp rx=0 tx=0).stop 2024-09-16T00:40:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb5ffff640 1 -- 172.21.15.31:0/1638843070 reap_dead start 2024-09-16T00:40:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 shutdown_connections 2024-09-16T00:40:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 >> 172.21.15.31:0/1638843070 conn(0x7feb600fc7d0 msgr2=0x7feb600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 shutdown_connections 2024-09-16T00:40:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.733+0000 7feb65a90640 1 -- 172.21.15.31:0/1638843070 wait complete. 2024-09-16T00:40:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.025+0000 7fb33aefa640 1 -- 172.21.15.31:0/941669568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334102790 msgr2=0x7fb3341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.025+0000 7fb33aefa640 1 --2- 172.21.15.31:0/941669568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334102790 0x7fb3341067e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb324009ca0 tx=0x7fb32402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.025+0000 7fb33aefa640 1 -- 172.21.15.31:0/941669568 shutdown_connections 2024-09-16T00:40:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.025+0000 7fb33aefa640 1 --2- 172.21.15.31:0/941669568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb334106f10 0x7fb33410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.025+0000 7fb33aefa640 1 --2- 172.21.15.31:0/941669568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334102790 0x7fb3341067e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.025+0000 7fb33aefa640 1 --2- 172.21.15.31:0/941669568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb334101de0 0x7fb3341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.025+0000 7fb33aefa640 1 -- 172.21.15.31:0/941669568 >> 172.21.15.31:0/941669568 conn(0x7fb3340fd8c0 msgr2=0x7fb3340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.025+0000 7fb33aefa640 1 -- 172.21.15.31:0/941669568 shutdown_connections 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.025+0000 7fb33aefa640 1 -- 172.21.15.31:0/941669568 wait complete. 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb33aefa640 1 Processor -- start 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb33aefa640 1 -- start start 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb33aefa640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb334101de0 0x7fb334198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb33aefa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb334102790 0x7fb3341989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb33aefa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334106f10 0x7fb33419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb33aefa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb334111f70 con 0x7fb334106f10 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb33aefa640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb334111df0 con 0x7fb334102790 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb33aefa640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb3341120f0 con 0x7fb334101de0 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb339497640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334106f10 0x7fb33419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb339497640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334106f10 0x7fb33419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50556/0 (socket says 172.21.15.31:50556) 2024-09-16T00:40:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb339497640 1 -- 172.21.15.31:0/1613669122 learned_addr learned my addr 172.21.15.31:0/1613669122 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb333fff640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb334102790 0x7fb3341989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb338c96640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb334101de0 0x7fb334198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb339497640 1 -- 172.21.15.31:0/1613669122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb334101de0 msgr2=0x7fb334198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb339497640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb334101de0 0x7fb334198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb339497640 1 -- 172.21.15.31:0/1613669122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb334102790 msgr2=0x7fb3341989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb339497640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb334102790 0x7fb3341989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb339497640 1 -- 172.21.15.31:0/1613669122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33419d420 con 0x7fb334106f10 2024-09-16T00:40:09.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.026+0000 7fb338c96640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb334101de0 0x7fb334198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:09.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.027+0000 7fb339497640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334106f10 0x7fb33419cd40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c970 tx=0x7fb32c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.027+0000 7fb331ffb640 1 -- 172.21.15.31:0/1613669122 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c007bb0 con 0x7fb334106f10 2024-09-16T00:40:09.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.027+0000 7fb331ffb640 1 -- 172.21.15.31:0/1613669122 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb32c007d50 con 0x7fb334106f10 2024-09-16T00:40:09.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.027+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33419d710 con 0x7fb334106f10 2024-09-16T00:40:09.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.027+0000 7fb331ffb640 1 -- 172.21.15.31:0/1613669122 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c00f450 con 0x7fb334106f10 2024-09-16T00:40:09.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.027+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3341a4ff0 con 0x7fb334106f10 2024-09-16T00:40:09.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.028+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb334062980 con 0x7fb334106f10 2024-09-16T00:40:09.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.029+0000 7fb331ffb640 1 -- 172.21.15.31:0/1613669122 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb32c004f20 con 0x7fb334106f10 2024-09-16T00:40:09.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.030+0000 7fb331ffb640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3200a3350 0x7fb3200a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.030+0000 7fb338c96640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3200a3350 0x7fb3200a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.030+0000 7fb331ffb640 1 -- 172.21.15.31:0/1613669122 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fb32c0cf9c0 con 0x7fb334106f10 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.030+0000 7fb331ffb640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] conn(0x7fb3200ae420 0x7fb3200b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.030+0000 7fb331ffb640 1 -- 172.21.15.31:0/1613669122 --> [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb32c004490 con 0x7fb3200ae420 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.030+0000 7fb331ffb640 1 -- 172.21.15.31:0/1613669122 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb32c0d42a0 con 0x7fb334106f10 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.030+0000 7fb333fff640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] conn(0x7fb3200ae420 0x7fb3200b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.030+0000 7fb338c96640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3200a3350 0x7fb3200a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0058c0 tx=0x7fb31c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.031+0000 7fb333fff640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] conn(0x7fb3200ae420 0x7fb3200b08c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.031+0000 7fb331ffb640 1 -- 172.21.15.31:0/1613669122 <== osd.0 v2:172.21.15.31:6802/1435627868 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb32c004490 con 0x7fb3200ae420 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.045+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 --> [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb334062e30 con 0x7fb3200ae420 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.045+0000 7fb331ffb640 1 -- 172.21.15.31:0/1613669122 <== osd.0 v2:172.21.15.31:6802/1435627868 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb334062e30 con 0x7fb3200ae420 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.045+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] conn(0x7fb3200ae420 msgr2=0x7fb3200b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.045+0000 7fb33aefa640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] conn(0x7fb3200ae420 0x7fb3200b08c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.046+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3200a3350 msgr2=0x7fb3200a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.046+0000 7fb33aefa640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3200a3350 0x7fb3200a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0058c0 tx=0x7fb31c005850 comp rx=0 tx=0).stop 2024-09-16T00:40:09.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.046+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334106f10 msgr2=0x7fb33419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.046+0000 7fb33aefa640 1 --2- 172.21.15.31:0/1613669122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334106f10 0x7fb33419cd40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c970 tx=0x7fb32c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:40:09.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.046+0000 7fb339497640 1 -- 172.21.15.31:0/1613669122 reap_dead start 2024-09-16T00:40:09.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.046+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 shutdown_connections 2024-09-16T00:40:09.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.046+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 >> 172.21.15.31:0/1613669122 conn(0x7fb3340fd8c0 msgr2=0x7fb3340fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.046+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 shutdown_connections 2024-09-16T00:40:09.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:09.046+0000 7fb33aefa640 1 -- 172.21.15.31:0/1613669122 wait complete. 2024-09-16T00:40:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.912+0000 7f9518d6a640 1 -- 172.21.15.31:0/2840177700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9514102790 msgr2=0x7f95141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.912+0000 7f9518d6a640 1 --2- 172.21.15.31:0/2840177700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9514102790 0x7f95141067e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9508009ca0 tx=0x7f950802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.912+0000 7f9518d6a640 1 -- 172.21.15.31:0/2840177700 shutdown_connections 2024-09-16T00:40:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.912+0000 7f9518d6a640 1 --2- 172.21.15.31:0/2840177700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9514106f10 0x7f951410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.912+0000 7f9518d6a640 1 --2- 172.21.15.31:0/2840177700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9514102790 0x7f95141067e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.912+0000 7f9518d6a640 1 --2- 172.21.15.31:0/2840177700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9514101de0 0x7f95141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.912+0000 7f9518d6a640 1 -- 172.21.15.31:0/2840177700 >> 172.21.15.31:0/2840177700 conn(0x7f95140fd8c0 msgr2=0x7f95140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.912+0000 7f9518d6a640 1 -- 172.21.15.31:0/2840177700 shutdown_connections 2024-09-16T00:40:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.912+0000 7f9518d6a640 1 -- 172.21.15.31:0/2840177700 wait complete. 2024-09-16T00:40:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9518d6a640 1 Processor -- start 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9518d6a640 1 -- start start 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9518d6a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9514101de0 0x7f951419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9518d6a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9514102790 0x7f951419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9518d6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9514106f10 0x7f951419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9518d6a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9514111f30 con 0x7f9514106f10 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9518d6a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9514111db0 con 0x7f9514102790 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9518d6a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f95141120b0 con 0x7f9514101de0 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9512d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9514106f10 0x7f951419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9512d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9514106f10 0x7f951419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50534/0 (socket says 172.21.15.31:50534) 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9512d9d640 1 -- 172.21.15.31:0/3314845454 learned_addr learned my addr 172.21.15.31:0/3314845454 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:09.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f9511d9b640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9514102790 0x7f951419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.913+0000 7f951259c640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9514101de0 0x7f951419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f9512d9d640 1 -- 172.21.15.31:0/3314845454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9514101de0 msgr2=0x7f951419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f9512d9d640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9514101de0 0x7f951419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f9512d9d640 1 -- 172.21.15.31:0/3314845454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9514102790 msgr2=0x7f951419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f9512d9d640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9514102790 0x7f951419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f9512d9d640 1 -- 172.21.15.31:0/3314845454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951419f510 con 0x7f9514106f10 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f951259c640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9514101de0 0x7f951419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f9512d9d640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9514106f10 0x7f951419ee30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f950c00c970 tx=0x7f950c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f95037fe640 1 -- 172.21.15.31:0/3314845454 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c007bb0 con 0x7f9514106f10 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951419f800 con 0x7f9514106f10 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f95037fe640 1 -- 172.21.15.31:0/3314845454 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950c007d50 con 0x7f9514106f10 2024-09-16T00:40:09.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f95037fe640 1 -- 172.21.15.31:0/3314845454 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c018800 con 0x7f9514106f10 2024-09-16T00:40:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.914+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95141a70e0 con 0x7f9514106f10 2024-09-16T00:40:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.915+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f95140feeb0 con 0x7f9514106f10 2024-09-16T00:40:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.917+0000 7f95037fe640 1 -- 172.21.15.31:0/3314845454 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f950c00fa50 con 0x7f9514106f10 2024-09-16T00:40:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.917+0000 7f95037fe640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f94fc0a32d0 0x7f94fc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.917+0000 7f95037fe640 1 -- 172.21.15.31:0/3314845454 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f950c0d0200 con 0x7f9514106f10 2024-09-16T00:40:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.917+0000 7f951259c640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f94fc0a32d0 0x7f94fc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.917+0000 7f95037fe640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] conn(0x7f94fc0ae3a0 0x7f94fc0b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.917+0000 7f95037fe640 1 -- 172.21.15.31:0/3314845454 --> [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f950c00eec0 con 0x7f94fc0ae3a0 2024-09-16T00:40:09.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.918+0000 7f95037fe640 1 -- 172.21.15.31:0/3314845454 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f950c0d0650 con 0x7f9514106f10 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.918+0000 7f951259c640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f94fc0a32d0 0x7f94fc0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f950400abe0 tx=0x7f950400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.918+0000 7f9511d9b640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] conn(0x7f94fc0ae3a0 0x7f94fc0b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.918+0000 7f9511d9b640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] conn(0x7f94fc0ae3a0 0x7f94fc0b0840 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.919+0000 7f95037fe640 1 -- 172.21.15.31:0/3314845454 <== osd.11 v2:172.21.15.102:6824/1575052259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f950c00eec0 con 0x7f94fc0ae3a0 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.933+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 --> [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f95141032c0 con 0x7f94fc0ae3a0 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.933+0000 7f95037fe640 1 -- 172.21.15.31:0/3314845454 <== osd.11 v2:172.21.15.102:6824/1575052259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f95141032c0 con 0x7f94fc0ae3a0 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 >> [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] conn(0x7f94fc0ae3a0 msgr2=0x7f94fc0b0840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] conn(0x7f94fc0ae3a0 0x7f94fc0b0840 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f94fc0a32d0 msgr2=0x7f94fc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f94fc0a32d0 0x7f94fc0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f950400abe0 tx=0x7f950400a5c0 comp rx=0 tx=0).stop 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9514106f10 msgr2=0x7f951419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:09.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 --2- 172.21.15.31:0/3314845454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9514106f10 0x7f951419ee30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f950c00c970 tx=0x7f950c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:40:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9512d9d640 1 -- 172.21.15.31:0/3314845454 reap_dead start 2024-09-16T00:40:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 shutdown_connections 2024-09-16T00:40:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 >> 172.21.15.31:0/3314845454 conn(0x7f95140fd8c0 msgr2=0x7f95140ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 shutdown_connections 2024-09-16T00:40:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:08.934+0000 7f9518d6a640 1 -- 172.21.15.31:0/3314845454 wait complete. 2024-09-16T00:40:09.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:09 smithi031 ceph-mon[30956]: pgmap v324: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 12 B/s, 0 objects/s recovering 2024-09-16T00:40:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:09 smithi081 ceph-mon[35539]: pgmap v324: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 12 B/s, 0 objects/s recovering 2024-09-16T00:40:09.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:09 smithi102 ceph-mon[35364]: pgmap v324: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 12 B/s, 0 objects/s recovering 2024-09-16T00:40:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:73014444107 2024-09-16T00:40:10.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T00:40:10.706 INFO:teuthology.orchestra.run.smithi031.stdout:188978561079 2024-09-16T00:40:10.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T00:40:11.199 INFO:teuthology.orchestra.run.smithi031.stdout:322122547229 2024-09-16T00:40:11.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-16T00:40:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.070+0000 7f903693e640 1 -- 172.21.15.31:0/4163704303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9030100e20 msgr2=0x7f9030101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.070+0000 7f903693e640 1 --2- 172.21.15.31:0/4163704303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9030100e20 0x7f9030101200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9020009900 tx=0x7f902002a480 comp rx=0 tx=0).stop 2024-09-16T00:40:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.070+0000 7f903693e640 1 -- 172.21.15.31:0/4163704303 shutdown_connections 2024-09-16T00:40:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.070+0000 7f903693e640 1 --2- 172.21.15.31:0/4163704303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9030102ef0 0x7f903010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.070+0000 7f903693e640 1 --2- 172.21.15.31:0/4163704303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9030102570 0x7f90301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.070+0000 7f903693e640 1 --2- 172.21.15.31:0/4163704303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9030100e20 0x7f9030101200 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.070+0000 7f903693e640 1 -- 172.21.15.31:0/4163704303 >> 172.21.15.31:0/4163704303 conn(0x7f90300fc7d0 msgr2=0x7f90300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.070+0000 7f903693e640 1 -- 172.21.15.31:0/4163704303 shutdown_connections 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.070+0000 7f903693e640 1 -- 172.21.15.31:0/4163704303 wait complete. 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f903693e640 1 Processor -- start 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f903693e640 1 -- start start 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f903693e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9030100e20 0x7f9030198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f903693e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9030102570 0x7f90301989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f903693e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9030102ef0 0x7f903019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f903693e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9030110f80 con 0x7f9030100e20 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f903693e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9030110e00 con 0x7f9030102ef0 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f903693e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9030111100 con 0x7f9030102570 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f902ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9030100e20 0x7f9030198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f902ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9030100e20 0x7f9030198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43192/0 (socket says 172.21.15.31:43192) 2024-09-16T00:40:11.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f902ffff640 1 -- 172.21.15.31:0/2322509392 learned_addr learned my addr 172.21.15.31:0/2322509392 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:11.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f9027fff640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9030102570 0x7f90301989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:11.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f9034edb640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9030102ef0 0x7f903019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:11.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f902ffff640 1 -- 172.21.15.31:0/2322509392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9030102570 msgr2=0x7f90301989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:11.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.071+0000 7f902ffff640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9030102570 0x7f90301989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:11.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f902ffff640 1 -- 172.21.15.31:0/2322509392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9030102ef0 msgr2=0x7f903019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:11.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f902ffff640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9030102ef0 0x7f903019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:11.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f902ffff640 1 -- 172.21.15.31:0/2322509392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903019d440 con 0x7f9030100e20 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f9027fff640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9030102570 0x7f90301989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f9034edb640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9030102ef0 0x7f903019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f902ffff640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9030100e20 0x7f9030198490 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9020006010 tx=0x7f9020003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f902dffb640 1 -- 172.21.15.31:0/2322509392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902003f070 con 0x7f9030100e20 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f902dffb640 1 -- 172.21.15.31:0/2322509392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9020031e30 con 0x7f9030100e20 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f902dffb640 1 -- 172.21.15.31:0/2322509392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020035030 con 0x7f9030100e20 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903019d6d0 con 0x7f9030100e20 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.072+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90301a5010 con 0x7f9030100e20 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.073+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9030106bf0 con 0x7f9030100e20 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.074+0000 7f902dffb640 1 -- 172.21.15.31:0/2322509392 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9020042070 con 0x7f9030100e20 2024-09-16T00:40:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.075+0000 7f902dffb640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f901c0a32d0 0x7f901c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.075+0000 7f902dffb640 1 -- 172.21.15.31:0/2322509392 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f90200f0de0 con 0x7f9030100e20 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.075+0000 7f9027fff640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f901c0a32d0 0x7f901c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.075+0000 7f902dffb640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] conn(0x7f901c0ae390 0x7f901c0b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.075+0000 7f902dffb640 1 -- 172.21.15.31:0/2322509392 --> [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9020038e90 con 0x7f901c0ae390 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.075+0000 7f902dffb640 1 -- 172.21.15.31:0/2322509392 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f90200f1230 con 0x7f9030100e20 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.075+0000 7f9034edb640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] conn(0x7f901c0ae390 0x7f901c0b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.075+0000 7f9027fff640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f901c0a32d0 0x7f901c0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f90300ffee0 tx=0x7f9018005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.076+0000 7f9034edb640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] conn(0x7f901c0ae390 0x7f901c0b0830 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.077+0000 7f902dffb640 1 -- 172.21.15.31:0/2322509392 <== osd.4 v2:172.21.15.81:6808/59224774 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9020038e90 con 0x7f901c0ae390 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.091+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 --> [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f90301021b0 con 0x7f901c0ae390 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f902dffb640 1 -- 172.21.15.31:0/2322509392 <== osd.4 v2:172.21.15.81:6808/59224774 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f90301021b0 con 0x7f901c0ae390 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 >> [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] conn(0x7f901c0ae390 msgr2=0x7f901c0b0830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:11.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] conn(0x7f901c0ae390 0x7f901c0b0830 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f901c0a32d0 msgr2=0x7f901c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f901c0a32d0 0x7f901c0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f90300ffee0 tx=0x7f9018005d30 comp rx=0 tx=0).stop 2024-09-16T00:40:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9030100e20 msgr2=0x7f9030198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 --2- 172.21.15.31:0/2322509392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9030100e20 0x7f9030198490 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9020006010 tx=0x7f9020003c80 comp rx=0 tx=0).stop 2024-09-16T00:40:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f9034edb640 1 -- 172.21.15.31:0/2322509392 reap_dead start 2024-09-16T00:40:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 shutdown_connections 2024-09-16T00:40:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 >> 172.21.15.31:0/2322509392 conn(0x7f90300fc7d0 msgr2=0x7f903006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 shutdown_connections 2024-09-16T00:40:11.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:06.092+0000 7f903693e640 1 -- 172.21.15.31:0/2322509392 wait complete. 2024-09-16T00:40:11.428 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:11 smithi031 ceph-mon[30956]: pgmap v325: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 994 B/s rd, 1 op/s; 11 B/s, 0 objects/s recovering 2024-09-16T00:40:11.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:11 smithi081 ceph-mon[35539]: pgmap v325: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 994 B/s rd, 1 op/s; 11 B/s, 0 objects/s recovering 2024-09-16T00:40:11.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:11 smithi102 ceph-mon[35364]: pgmap v325: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 994 B/s rd, 1 op/s; 11 B/s, 0 objects/s recovering 2024-09-16T00:40:12.169 INFO:teuthology.orchestra.run.smithi031.stdout:55834574927 2024-09-16T00:40:12.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T00:40:12.660 INFO:teuthology.orchestra.run.smithi031.stdout:270582939685 2024-09-16T00:40:12.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-16T00:40:12.992 INFO:teuthology.orchestra.run.smithi031.stdout:34359738453 2024-09-16T00:40:12.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T00:40:13.421 INFO:teuthology.orchestra.run.smithi031.stdout:137438953536 2024-09-16T00:40:13.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T00:40:13.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:13 smithi081 ceph-mon[35539]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:40:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:13 smithi102 ceph-mon[35364]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:40:13.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:13 smithi031 ceph-mon[30956]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:40:13.927 INFO:teuthology.orchestra.run.smithi031.stdout:240518168620 2024-09-16T00:40:13.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-16T00:40:13.938 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1499291573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8102570 msgr2=0x7f96c81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1499291573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8102570 0x7f96c81029b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009900 tx=0x7f96b002a480 comp rx=0 tx=0).stop 2024-09-16T00:40:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1499291573 shutdown_connections 2024-09-16T00:40:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1499291573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c8102ef0 0x7f96c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1499291573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8102570 0x7f96c81029b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1499291573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96c8100e20 0x7f96c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1499291573 >> 172.21.15.31:0/1499291573 conn(0x7f96c80fc7d0 msgr2=0x7f96c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1499291573 shutdown_connections 2024-09-16T00:40:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1499291573 wait complete. 2024-09-16T00:40:14.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.491+0000 7f96cd7a5640 1 Processor -- start 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96cd7a5640 1 -- start start 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96cd7a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8100e20 0x7f96c81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96cd7a5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96c8102570 0x7f96c8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96cd7a5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c8102ef0 0x7f96c81a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96cd7a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96c8110f50 con 0x7f96c8100e20 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96cd7a5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f96c8110dd0 con 0x7f96c8102570 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96cd7a5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f96c81110d0 con 0x7f96c8102ef0 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8100e20 0x7f96c81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8100e20 0x7f96c81992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50582/0 (socket says 172.21.15.31:50582) 2024-09-16T00:40:14.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c6ffd640 1 -- 172.21.15.31:0/1127235565 learned_addr learned my addr 172.21.15.31:0/1127235565 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c67fc640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96c8102570 0x7f96c8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c77fe640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c8102ef0 0x7f96c81a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c6ffd640 1 -- 172.21.15.31:0/1127235565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c8102ef0 msgr2=0x7f96c81a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c6ffd640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c8102ef0 0x7f96c81a4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c6ffd640 1 -- 172.21.15.31:0/1127235565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96c8102570 msgr2=0x7f96c8199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c6ffd640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96c8102570 0x7f96c8199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c6ffd640 1 -- 172.21.15.31:0/1127235565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c81a48a0 con 0x7f96c8100e20 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c67fc640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96c8102570 0x7f96c8199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.492+0000 7f96c6ffd640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8100e20 0x7f96c81992e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f96c000cce0 tx=0x7f96c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.493+0000 7f96bffff640 1 -- 172.21.15.31:0/1127235565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000f650 con 0x7f96c8100e20 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.493+0000 7f96bffff640 1 -- 172.21.15.31:0/1127235565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c000fcd0 con 0x7f96c8100e20 2024-09-16T00:40:14.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.493+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c81a4a70 con 0x7f96c8100e20 2024-09-16T00:40:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.493+0000 7f96bffff640 1 -- 172.21.15.31:0/1127235565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0018c70 con 0x7f96c8100e20 2024-09-16T00:40:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.493+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c81a4f80 con 0x7f96c8100e20 2024-09-16T00:40:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.494+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c8101200 con 0x7f96c8100e20 2024-09-16T00:40:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.495+0000 7f96bffff640 1 -- 172.21.15.31:0/1127235565 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f96c000f7f0 con 0x7f96c8100e20 2024-09-16T00:40:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.496+0000 7f96bffff640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96b40a32d0 0x7f96b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.496+0000 7f96c67fc640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96b40a32d0 0x7f96b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.496+0000 7f96bffff640 1 -- 172.21.15.31:0/1127235565 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f96c00d0330 con 0x7f96c8100e20 2024-09-16T00:40:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.496+0000 7f96c67fc640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96b40a32d0 0x7f96b40a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f96c806aac0 tx=0x7f96b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:14.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.501+0000 7f96bffff640 1 -- 172.21.15.31:0/1127235565 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96c0014030 con 0x7f96c8100e20 2024-09-16T00:40:14.623 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.637 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.637 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.638 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.642 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.643 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.647 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.648 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.660 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.733+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f96c81a5210 con 0x7f96c8100e20 2024-09-16T00:40:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.734+0000 7f96bffff640 1 -- 172.21.15.31:0/1127235565 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f96c009cbc0 con 0x7f96c8100e20 2024-09-16T00:40:14.734 INFO:teuthology.orchestra.run.smithi031.stdout:210453397555 2024-09-16T00:40:14.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96b40a32d0 msgr2=0x7f96b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96b40a32d0 0x7f96b40a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f96c806aac0 tx=0x7f96b0005a20 comp rx=0 tx=0).stop 2024-09-16T00:40:14.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8100e20 msgr2=0x7f96c81992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8100e20 0x7f96c81992e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f96c000cce0 tx=0x7f96c000a620 comp rx=0 tx=0).stop 2024-09-16T00:40:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 shutdown_connections 2024-09-16T00:40:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96b40a32d0 0x7f96b40a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96c8102ef0 0x7f96c81a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96c8102570 0x7f96c8199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 --2- 172.21.15.31:0/1127235565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96c8100e20 0x7f96c81992e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 >> 172.21.15.31:0/1127235565 conn(0x7f96c80fc7d0 msgr2=0x7f96c8106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 shutdown_connections 2024-09-16T00:40:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:14.736+0000 7f96cd7a5640 1 -- 172.21.15.31:0/1127235565 wait complete. 2024-09-16T00:40:15.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.603+0000 7f415b5ed640 1 -- 172.21.15.31:0/1152146464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 msgr2=0x7f41541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:15.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.603+0000 7f415b5ed640 1 --2- 172.21.15.31:0/1152146464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 0x7f41541029b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4150009900 tx=0x7f415002a480 comp rx=0 tx=0).stop 2024-09-16T00:40:15.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.603+0000 7f415b5ed640 1 -- 172.21.15.31:0/1152146464 shutdown_connections 2024-09-16T00:40:15.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.603+0000 7f415b5ed640 1 --2- 172.21.15.31:0/1152146464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4154102ef0 0x7f415410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.603+0000 7f415b5ed640 1 --2- 172.21.15.31:0/1152146464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 0x7f41541029b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.603+0000 7f415b5ed640 1 --2- 172.21.15.31:0/1152146464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4154100e20 0x7f4154101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.603+0000 7f415b5ed640 1 -- 172.21.15.31:0/1152146464 >> 172.21.15.31:0/1152146464 conn(0x7f41540fc7d0 msgr2=0x7f41540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.603+0000 7f415b5ed640 1 -- 172.21.15.31:0/1152146464 shutdown_connections 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.603+0000 7f415b5ed640 1 -- 172.21.15.31:0/1152146464 wait complete. 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f415b5ed640 1 Processor -- start 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f415b5ed640 1 -- start start 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f415b5ed640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4154100e20 0x7f4154199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f415b5ed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 0x7f41541998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f415b5ed640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4154102ef0 0x7f41541a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f415b5ed640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4154110f80 con 0x7f4154102570 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f415b5ed640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4154110e00 con 0x7f4154102ef0 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f415b5ed640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4154111100 con 0x7f4154100e20 2024-09-16T00:40:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4158b88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 0x7f41541998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:15.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4158b88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 0x7f41541998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50594/0 (socket says 172.21.15.31:50594) 2024-09-16T00:40:15.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4158b88640 1 -- 172.21.15.31:0/2987113104 learned_addr learned my addr 172.21.15.31:0/2987113104 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:15.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4159389640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4154100e20 0x7f4154199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:15.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4159b8a640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4154102ef0 0x7f41541a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:15.684 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:15 smithi031 ceph-mon[30956]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:40:15.684 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:15 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1127235565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4158b88640 1 -- 172.21.15.31:0/2987113104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4154100e20 msgr2=0x7f4154199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4158b88640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4154100e20 0x7f4154199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4158b88640 1 -- 172.21.15.31:0/2987113104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4154102ef0 msgr2=0x7f41541a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4158b88640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4154102ef0 0x7f41541a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.604+0000 7f4158b88640 1 -- 172.21.15.31:0/2987113104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41541a4930 con 0x7f4154102570 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.605+0000 7f4158b88640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 0x7f41541998b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4150006010 tx=0x7f4150003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.605+0000 7f41467fc640 1 -- 172.21.15.31:0/2987113104 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415003f070 con 0x7f4154102570 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.605+0000 7f41467fc640 1 -- 172.21.15.31:0/2987113104 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4150031c30 con 0x7f4154102570 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.605+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41541a4b00 con 0x7f4154102570 2024-09-16T00:40:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.605+0000 7f41467fc640 1 -- 172.21.15.31:0/2987113104 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150035030 con 0x7f4154102570 2024-09-16T00:40:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.605+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41541a4f50 con 0x7f4154102570 2024-09-16T00:40:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.606+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4154101200 con 0x7f4154102570 2024-09-16T00:40:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.607+0000 7f41467fc640 1 -- 172.21.15.31:0/2987113104 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4150042070 con 0x7f4154102570 2024-09-16T00:40:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.608+0000 7f41467fc640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41400a32d0 0x7f41400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.608+0000 7f4159389640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41400a32d0 0x7f41400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.608+0000 7f41467fc640 1 -- 172.21.15.31:0/2987113104 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f41500f0900 con 0x7f4154102570 2024-09-16T00:40:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.608+0000 7f4159389640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41400a32d0 0x7f41400a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4148009bd0 tx=0x7f4148005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.611+0000 7f41467fc640 1 -- 172.21.15.31:0/2987113104 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41500bcb40 con 0x7f4154102570 2024-09-16T00:40:15.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:15 smithi081 ceph-mon[35539]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:40:15.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:15 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1127235565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:40:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.791+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f4154109090 con 0x7f4154102570 2024-09-16T00:40:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.792+0000 7f41467fc640 1 -- 172.21.15.31:0/2987113104 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f41500c0e40 con 0x7f4154102570 2024-09-16T00:40:15.792 INFO:teuthology.orchestra.run.smithi031.stdout:270582939687 2024-09-16T00:40:15.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41400a32d0 msgr2=0x7f41400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:15.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41400a32d0 0x7f41400a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4148009bd0 tx=0x7f4148005f90 comp rx=0 tx=0).stop 2024-09-16T00:40:15.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 msgr2=0x7f41541998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:15.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 0x7f41541998b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4150006010 tx=0x7f4150003c10 comp rx=0 tx=0).stop 2024-09-16T00:40:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 shutdown_connections 2024-09-16T00:40:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4154102ef0 0x7f41541a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4154102570 0x7f41541998b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41400a32d0 0x7f41400a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 --2- 172.21.15.31:0/2987113104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4154100e20 0x7f4154199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 >> 172.21.15.31:0/2987113104 conn(0x7f41540fc7d0 msgr2=0x7f41540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 shutdown_connections 2024-09-16T00:40:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.794+0000 7f415b5ed640 1 -- 172.21.15.31:0/2987113104 wait complete. 2024-09-16T00:40:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:15 smithi102 ceph-mon[35364]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:40:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:15 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1127235565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:40:15.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.882+0000 7fec7ddcb640 1 -- 172.21.15.31:0/981340074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec781062b0 msgr2=0x7fec78106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:15.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.882+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/981340074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec781062b0 0x7fec78106690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fec68009930 tx=0x7fec6802a420 comp rx=0 tx=0).stop 2024-09-16T00:40:15.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.882+0000 7fec7ddcb640 1 -- 172.21.15.31:0/981340074 shutdown_connections 2024-09-16T00:40:15.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.882+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/981340074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec78100c00 0x7fec7810d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.882+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/981340074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec781002a0 0x7fec781006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.882+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/981340074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec781062b0 0x7fec78106690 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.882+0000 7fec7ddcb640 1 -- 172.21.15.31:0/981340074 >> 172.21.15.31:0/981340074 conn(0x7fec780fc5c0 msgr2=0x7fec780fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.882+0000 7fec7ddcb640 1 -- 172.21.15.31:0/981340074 shutdown_connections 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.882+0000 7fec7ddcb640 1 -- 172.21.15.31:0/981340074 wait complete. 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec7ddcb640 1 Processor -- start 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec7ddcb640 1 -- start start 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec7ddcb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec781002a0 0x7fec78198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec7ddcb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec78100c00 0x7fec781989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec7ddcb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec781062b0 0x7fec7819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec7ddcb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810fd90 con 0x7fec78100c00 2024-09-16T00:40:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec7ddcb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810fc10 con 0x7fec781002a0 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec7ddcb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810ff10 con 0x7fec781062b0 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec76ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec78100c00 0x7fec781989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec76ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec78100c00 0x7fec781989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50612/0 (socket says 172.21.15.31:50612) 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec76ffd640 1 -- 172.21.15.31:0/2714496247 learned_addr learned my addr 172.21.15.31:0/2714496247 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec77fff640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec781062b0 0x7fec7819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.883+0000 7fec777fe640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec781002a0 0x7fec78198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec76ffd640 1 -- 172.21.15.31:0/2714496247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec781062b0 msgr2=0x7fec7819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec76ffd640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec781062b0 0x7fec7819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec76ffd640 1 -- 172.21.15.31:0/2714496247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec781002a0 msgr2=0x7fec78198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec76ffd640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec781002a0 0x7fec78198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec76ffd640 1 -- 172.21.15.31:0/2714496247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7819d440 con 0x7fec78100c00 2024-09-16T00:40:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec777fe640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec781002a0 0x7fec78198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec76ffd640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec78100c00 0x7fec781989d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00dbc0 tx=0x7fec6c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec74ff9640 1 -- 172.21.15.31:0/2714496247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c005220 con 0x7fec78100c00 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec74ff9640 1 -- 172.21.15.31:0/2714496247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec6c00ae50 con 0x7fec78100c00 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7819d730 con 0x7fec78100c00 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec74ff9640 1 -- 172.21.15.31:0/2714496247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00e870 con 0x7fec78100c00 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.884+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec781a5010 con 0x7fec78100c00 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.885+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec781006c0 con 0x7fec78100c00 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.887+0000 7fec74ff9640 1 -- 172.21.15.31:0/2714496247 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fec6c00ea10 con 0x7fec78100c00 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.887+0000 7fec74ff9640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fec600a32d0 0x7fec600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.888+0000 7fec777fe640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fec600a32d0 0x7fec600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.888+0000 7fec74ff9640 1 -- 172.21.15.31:0/2714496247 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fec6c0d0220 con 0x7fec78100c00 2024-09-16T00:40:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.888+0000 7fec777fe640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fec600a32d0 0x7fec600a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec68000c00 tx=0x7fec68005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:15.891+0000 7fec74ff9640 1 -- 172.21.15.31:0/2714496247 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec6c09cbc0 con 0x7fec78100c00 2024-09-16T00:40:16.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.068+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fec7810a780 con 0x7fec78100c00 2024-09-16T00:40:16.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.068+0000 7fec74ff9640 1 -- 172.21.15.31:0/2714496247 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fec6c09c4c0 con 0x7fec78100c00 2024-09-16T00:40:16.069 INFO:teuthology.orchestra.run.smithi031.stdout:158913790012 2024-09-16T00:40:16.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.070+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fec600a32d0 msgr2=0x7fec600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.070+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fec600a32d0 0x7fec600a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec68000c00 tx=0x7fec68005a20 comp rx=0 tx=0).stop 2024-09-16T00:40:16.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.070+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec78100c00 msgr2=0x7fec781989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:16.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.070+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec78100c00 0x7fec781989d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00dbc0 tx=0x7fec6c00a620 comp rx=0 tx=0).stop 2024-09-16T00:40:16.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.071+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 shutdown_connections 2024-09-16T00:40:16.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.071+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fec600a32d0 0x7fec600a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.071+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec781062b0 0x7fec7819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.071+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec78100c00 0x7fec781989d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.071+0000 7fec7ddcb640 1 --2- 172.21.15.31:0/2714496247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec781002a0 0x7fec78198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:16.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.071+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 >> 172.21.15.31:0/2714496247 conn(0x7fec780fc5c0 msgr2=0x7fec781046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:16.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.071+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 shutdown_connections 2024-09-16T00:40:16.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:16.071+0000 7fec7ddcb640 1 -- 172.21.15.31:0/2714496247 wait complete. 2024-09-16T00:40:16.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939685 got 270582939687 for osd.9 2024-09-16T00:40:16.551 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:16.676 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:16 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2987113104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:40:16.676 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:16 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2714496247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:40:16.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:16 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2987113104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:40:16.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:16 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2714496247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:40:16.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:16 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2987113104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:40:16.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:16 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2714496247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:40:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.552+0000 7fe17ceed640 1 -- 172.21.15.31:0/48155950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178101de0 msgr2=0x7fe1781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.552+0000 7fe17ceed640 1 --2- 172.21.15.31:0/48155950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178101de0 0x7fe1781021c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe168009900 tx=0x7fe16802a410 comp rx=0 tx=0).stop 2024-09-16T00:40:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.553+0000 7fe17ceed640 1 -- 172.21.15.31:0/48155950 shutdown_connections 2024-09-16T00:40:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.553+0000 7fe17ceed640 1 --2- 172.21.15.31:0/48155950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178106f10 0x7fe17810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.553+0000 7fe17ceed640 1 --2- 172.21.15.31:0/48155950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe178102790 0x7fe1781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.553+0000 7fe17ceed640 1 --2- 172.21.15.31:0/48155950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178101de0 0x7fe1781021c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.553+0000 7fe17ceed640 1 -- 172.21.15.31:0/48155950 >> 172.21.15.31:0/48155950 conn(0x7fe1780fd8c0 msgr2=0x7fe1780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.553+0000 7fe17ceed640 1 -- 172.21.15.31:0/48155950 shutdown_connections 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.553+0000 7fe17ceed640 1 -- 172.21.15.31:0/48155950 wait complete. 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.553+0000 7fe17ceed640 1 Processor -- start 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.553+0000 7fe17ceed640 1 -- start start 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe17ceed640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe178101de0 0x7fe178198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe17ceed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178102790 0x7fe1781989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe17ceed640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178106f10 0x7fe17819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe17ceed640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe178111fc0 con 0x7fe178102790 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe17ceed640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe178111e40 con 0x7fe178106f10 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe17ceed640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe178112140 con 0x7fe178101de0 2024-09-16T00:40:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe175d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178102790 0x7fe1781989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe175d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178102790 0x7fe1781989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47094/0 (socket says 172.21.15.31:47094) 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe175d9b640 1 -- 172.21.15.31:0/1399200316 learned_addr learned my addr 172.21.15.31:0/1399200316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe17659c640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe178101de0 0x7fe178198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe176d9d640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178106f10 0x7fe17819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe175d9b640 1 -- 172.21.15.31:0/1399200316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe178101de0 msgr2=0x7fe178198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe175d9b640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe178101de0 0x7fe178198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe175d9b640 1 -- 172.21.15.31:0/1399200316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178106f10 msgr2=0x7fe17819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe175d9b640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178106f10 0x7fe17819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe175d9b640 1 -- 172.21.15.31:0/1399200316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17819d430 con 0x7fe178102790 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe175d9b640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178102790 0x7fe1781989c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00e9f0 tx=0x7fe16c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe1677fe640 1 -- 172.21.15.31:0/1399200316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00cd80 con 0x7fe178102790 2024-09-16T00:40:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17819d720 con 0x7fe178102790 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe1677fe640 1 -- 172.21.15.31:0/1399200316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16c005500 con 0x7fe178102790 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe1677fe640 1 -- 172.21.15.31:0/1399200316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c010690 con 0x7fe178102790 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.554+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1781a5000 con 0x7fe178102790 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.555+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1781021c0 con 0x7fe178102790 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.557+0000 7fe1677fe640 1 -- 172.21.15.31:0/1399200316 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe16c0046c0 con 0x7fe178102790 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.557+0000 7fe1677fe640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1600a32d0 0x7fe1600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.557+0000 7fe17659c640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1600a32d0 0x7fe1600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.557+0000 7fe1677fe640 1 -- 172.21.15.31:0/1399200316 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fe16c0d0330 con 0x7fe178102790 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.558+0000 7fe17659c640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1600a32d0 0x7fe1600a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe168000c00 tx=0x7fe168005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.561+0000 7fe1677fe640 1 -- 172.21.15.31:0/1399200316 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe16c09cbc0 con 0x7fe178102790 2024-09-16T00:40:17.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:17 smithi031 ceph-mon[30956]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.739+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fe17802d7c0 con 0x7fe178102790 2024-09-16T00:40:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.739+0000 7fe1677fe640 1 -- 172.21.15.31:0/1399200316 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe16c09c4c0 con 0x7fe178102790 2024-09-16T00:40:17.740 INFO:teuthology.orchestra.run.smithi031.stdout:292057776164 2024-09-16T00:40:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1600a32d0 msgr2=0x7fe1600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1600a32d0 0x7fe1600a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe168000c00 tx=0x7fe168005a20 comp rx=0 tx=0).stop 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178102790 msgr2=0x7fe1781989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178102790 0x7fe1781989c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00e9f0 tx=0x7fe16c00edf0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 shutdown_connections 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178106f10 0x7fe17819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe178102790 0x7fe1781989c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1600a32d0 0x7fe1600a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 --2- 172.21.15.31:0/1399200316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe178101de0 0x7fe178198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 >> 172.21.15.31:0/1399200316 conn(0x7fe1780fd8c0 msgr2=0x7fe1780febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 shutdown_connections 2024-09-16T00:40:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.742+0000 7fe17ceed640 1 -- 172.21.15.31:0/1399200316 wait complete. 2024-09-16T00:40:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:17 smithi081 ceph-mon[35539]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:17.800 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:17 smithi102 ceph-mon[35364]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:18.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.790+0000 7fbcb9d23640 1 -- 172.21.15.31:0/4034953038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4102790 msgr2=0x7fbcb41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.790+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/4034953038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4102790 0x7fbcb41067e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009900 tx=0x7fbca802a410 comp rx=0 tx=0).stop 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.790+0000 7fbcb9d23640 1 -- 172.21.15.31:0/4034953038 shutdown_connections 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.790+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/4034953038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb4106f10 0x7fbcb410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.791+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/4034953038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4102790 0x7fbcb41067e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.791+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/4034953038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb4101de0 0x7fbcb41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.791+0000 7fbcb9d23640 1 -- 172.21.15.31:0/4034953038 >> 172.21.15.31:0/4034953038 conn(0x7fbcb40fd8c0 msgr2=0x7fbcb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.791+0000 7fbcb9d23640 1 -- 172.21.15.31:0/4034953038 shutdown_connections 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.791+0000 7fbcb9d23640 1 -- 172.21.15.31:0/4034953038 wait complete. 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.791+0000 7fbcb9d23640 1 Processor -- start 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.791+0000 7fbcb9d23640 1 -- start start 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb9d23640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4101de0 0x7fbcb41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb9d23640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb4102790 0x7fbcb41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb9d23640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb4106f10 0x7fbcb419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb9d23640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4111f70 con 0x7fbcb4101de0 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb9d23640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4111df0 con 0x7fbcb4106f10 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb9d23640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb41120f0 con 0x7fbcb4102790 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4101de0 0x7fbcb41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4101de0 0x7fbcb41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47114/0 (socket says 172.21.15.31:47114) 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb37fe640 1 -- 172.21.15.31:0/3317427902 learned_addr learned my addr 172.21.15.31:0/3317427902 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb2ffd640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb4102790 0x7fbcb41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb3fff640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb4106f10 0x7fbcb419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb37fe640 1 -- 172.21.15.31:0/3317427902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb4102790 msgr2=0x7fbcb41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb37fe640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb4102790 0x7fbcb41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb37fe640 1 -- 172.21.15.31:0/3317427902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb4106f10 msgr2=0x7fbcb419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb37fe640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb4106f10 0x7fbcb419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb37fe640 1 -- 172.21.15.31:0/3317427902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb419d460 con 0x7fbcb4101de0 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb37fe640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4101de0 0x7fbcb41984b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d950 tx=0x7fbca400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb0ff9640 1 -- 172.21.15.31:0/3317427902 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4017030 con 0x7fbcb4101de0 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.792+0000 7fbcb0ff9640 1 -- 172.21.15.31:0/3317427902 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca4004020 con 0x7fbcb4101de0 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.793+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb419d750 con 0x7fbcb4101de0 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.793+0000 7fbcb0ff9640 1 -- 172.21.15.31:0/3317427902 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca40047a0 con 0x7fbcb4101de0 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.793+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb41a5030 con 0x7fbcb4101de0 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.794+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb41021c0 con 0x7fbcb4101de0 2024-09-16T00:40:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.795+0000 7fbcb0ff9640 1 -- 172.21.15.31:0/3317427902 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbca40041d0 con 0x7fbcb4101de0 2024-09-16T00:40:18.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.796+0000 7fbcb0ff9640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc9c0a3350 0x7fbc9c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.796+0000 7fbcb2ffd640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc9c0a3350 0x7fbc9c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.796+0000 7fbcb0ff9640 1 -- 172.21.15.31:0/3317427902 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fbca40cfdd0 con 0x7fbcb4101de0 2024-09-16T00:40:18.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.796+0000 7fbcb2ffd640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc9c0a3350 0x7fbc9c0a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009230 tx=0x7fbca80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.443 INFO:teuthology.orchestra.run.smithi031.stdout:73014444109 2024-09-16T00:40:18.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.799+0000 7fbcb0ff9640 1 -- 172.21.15.31:0/3317427902 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbca409c6e0 con 0x7fbcb4101de0 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.977+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbcb4199810 con 0x7fbcb4101de0 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.978+0000 7fbcb0ff9640 1 -- 172.21.15.31:0/3317427902 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbcb4199810 con 0x7fbcb4101de0 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc9c0a3350 msgr2=0x7fbc9c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc9c0a3350 0x7fbc9c0a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009230 tx=0x7fbca80058d0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4101de0 msgr2=0x7fbcb41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4101de0 0x7fbcb41984b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d950 tx=0x7fbca400de10 comp rx=0 tx=0).stop 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 shutdown_connections 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb4106f10 0x7fbcb419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc9c0a3350 0x7fbc9c0a5810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb4102790 0x7fbcb41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 --2- 172.21.15.31:0/3317427902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcb4101de0 0x7fbcb41984b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.980+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 >> 172.21.15.31:0/3317427902 conn(0x7fbcb40fd8c0 msgr2=0x7fbcb40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.981+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 shutdown_connections 2024-09-16T00:40:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:17.981+0000 7fbcb9d23640 1 -- 172.21.15.31:0/3317427902 wait complete. 2024-09-16T00:40:18.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:18 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1399200316' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:40:18.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:18 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3317427902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:40:18.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:18 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1399200316' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:40:18.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:18 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3317427902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:40:18.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:18 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1399200316' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:40:18.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:18 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3317427902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:40:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.070+0000 7f34ce1bd640 1 -- 172.21.15.31:0/1923186084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8101de0 msgr2=0x7f34c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.070+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/1923186084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8101de0 0x7f34c81021c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009900 tx=0x7f34b802a410 comp rx=0 tx=0).stop 2024-09-16T00:40:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.070+0000 7f34ce1bd640 1 -- 172.21.15.31:0/1923186084 shutdown_connections 2024-09-16T00:40:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.070+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/1923186084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34c8106f10 0x7f34c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.070+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/1923186084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34c8102790 0x7f34c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.070+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/1923186084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8101de0 0x7f34c81021c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.070+0000 7f34ce1bd640 1 -- 172.21.15.31:0/1923186084 >> 172.21.15.31:0/1923186084 conn(0x7f34c80fd8c0 msgr2=0x7f34c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.070+0000 7f34ce1bd640 1 -- 172.21.15.31:0/1923186084 shutdown_connections 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.070+0000 7f34ce1bd640 1 -- 172.21.15.31:0/1923186084 wait complete. 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.071+0000 7f34ce1bd640 1 Processor -- start 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.071+0000 7f34ce1bd640 1 -- start start 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.071+0000 7f34ce1bd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34c8101de0 0x7f34c8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.071+0000 7f34ce1bd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34c8102790 0x7f34c8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.071+0000 7f34ce1bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8106f10 0x7f34c819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.071+0000 7f34ce1bd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8111f70 con 0x7f34c8106f10 2024-09-16T00:40:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.071+0000 7f34ce1bd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8111df0 con 0x7f34c8102790 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.071+0000 7f34ce1bd640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81120f0 con 0x7f34c8101de0 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.071+0000 7f34c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8106f10 0x7f34c819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8106f10 0x7f34c819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47134/0 (socket says 172.21.15.31:47134) 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34c8101de0 0x7f34c8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c7fff640 1 -- 172.21.15.31:0/660876079 learned_addr learned my addr 172.21.15.31:0/660876079 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34bffff640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34c8102790 0x7f34c8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c7fff640 1 -- 172.21.15.31:0/660876079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34c8101de0 msgr2=0x7f34c8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c7fff640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34c8101de0 0x7f34c8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c7fff640 1 -- 172.21.15.31:0/660876079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34c8102790 msgr2=0x7f34c8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c7fff640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34c8102790 0x7f34c8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c7fff640 1 -- 172.21.15.31:0/660876079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c819d400 con 0x7f34c8106f10 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c7fff640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8106f10 0x7f34c819cd20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f34c000eaa0 tx=0x7f34c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c57fa640 1 -- 172.21.15.31:0/660876079 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c000cdb0 con 0x7f34c8106f10 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c57fa640 1 -- 172.21.15.31:0/660876079 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c0005020 con 0x7f34c8106f10 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c819d6f0 con 0x7f34c8106f10 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34c57fa640 1 -- 172.21.15.31:0/660876079 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00116b0 con 0x7f34c8106f10 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.072+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c81a4fd0 con 0x7f34c8106f10 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.073+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c81021c0 con 0x7f34c8106f10 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.075+0000 7f34c57fa640 1 -- 172.21.15.31:0/660876079 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f34c0005560 con 0x7f34c8106f10 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.075+0000 7f34c57fa640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34b40a3350 0x7f34b40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.075+0000 7f34c57fa640 1 -- 172.21.15.31:0/660876079 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f34c00cfea0 con 0x7f34c8106f10 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.075+0000 7f34c77fe640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34b40a3350 0x7f34b40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.076+0000 7f34c77fe640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34b40a3350 0x7f34b40a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009230 tx=0x7f34b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.080+0000 7f34c57fa640 1 -- 172.21.15.31:0/660876079 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34c0016030 con 0x7f34c8106f10 2024-09-16T00:40:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.256+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f34c8109b60 con 0x7f34c8106f10 2024-09-16T00:40:19.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.256+0000 7f34c57fa640 1 -- 172.21.15.31:0/660876079 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f34c009c0e0 con 0x7f34c8106f10 2024-09-16T00:40:19.257 INFO:teuthology.orchestra.run.smithi031.stdout:188978561080 2024-09-16T00:40:19.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34b40a3350 msgr2=0x7f34b40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34b40a3350 0x7f34b40a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009230 tx=0x7f34b80058d0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8106f10 msgr2=0x7f34c819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8106f10 0x7f34c819cd20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f34c000eaa0 tx=0x7f34c000ef60 comp rx=0 tx=0).stop 2024-09-16T00:40:19.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 shutdown_connections 2024-09-16T00:40:19.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34c8106f10 0x7f34c819cd20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34c8102790 0x7f34c8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34b40a3350 0x7f34b40a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 --2- 172.21.15.31:0/660876079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34c8101de0 0x7f34c8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.258+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 >> 172.21.15.31:0/660876079 conn(0x7f34c80fd8c0 msgr2=0x7f34c80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.259+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 shutdown_connections 2024-09-16T00:40:19.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.259+0000 7f34ce1bd640 1 -- 172.21.15.31:0/660876079 wait complete. 2024-09-16T00:40:19.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:19 smithi031 ceph-mon[30956]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:19.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:19 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/660876079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:40:19.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:19 smithi081 ceph-mon[35539]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:19.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:19 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/660876079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:40:19.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 -- 172.21.15.31:0/4120050014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81082a0 msgr2=0x7f1ec810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 --2- 172.21.15.31:0/4120050014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81082a0 0x7f1ec810ec40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009cd0 tx=0x7f1ec002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 -- 172.21.15.31:0/4120050014 shutdown_connections 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 --2- 172.21.15.31:0/4120050014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81082a0 0x7f1ec810ec40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 --2- 172.21.15.31:0/4120050014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ec81036e0 0x7f1ec8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 --2- 172.21.15.31:0/4120050014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec8102d30 0x7f1ec8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 -- 172.21.15.31:0/4120050014 >> 172.21.15.31:0/4120050014 conn(0x7f1ec80fc7d0 msgr2=0x7f1ec80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 -- 172.21.15.31:0/4120050014 shutdown_connections 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 -- 172.21.15.31:0/4120050014 wait complete. 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.750+0000 7f1ecd434640 1 Processor -- start 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ecd434640 1 -- start start 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ecd434640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ec8102d30 0x7f1ec8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ecd434640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81036e0 0x7f1ec8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ecd434640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81082a0 0x7f1ec8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ecd434640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81120a0 con 0x7f1ec81036e0 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ecd434640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8111f20 con 0x7f1ec8102d30 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ecd434640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8112220 con 0x7f1ec81082a0 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ebe7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81036e0 0x7f1ec8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ebe7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81036e0 0x7f1ec8100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47158/0 (socket says 172.21.15.31:47158) 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ebe7fc640 1 -- 172.21.15.31:0/2669920432 learned_addr learned my addr 172.21.15.31:0/2669920432 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ec6ffd640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ec8102d30 0x7f1ec8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ec77fe640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81082a0 0x7f1ec8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ebe7fc640 1 -- 172.21.15.31:0/2669920432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81082a0 msgr2=0x7f1ec8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ebe7fc640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81082a0 0x7f1ec8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ebe7fc640 1 -- 172.21.15.31:0/2669920432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ec8102d30 msgr2=0x7f1ec8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ebe7fc640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ec8102d30 0x7f1ec8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ebe7fc640 1 -- 172.21.15.31:0/2669920432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81a6a30 con 0x7f1ec81036e0 2024-09-16T00:40:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.751+0000 7f1ebe7fc640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81036e0 0x7f1ec8100ed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000e9f0 tx=0x7f1eb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.752+0000 7f1ec4ff9640 1 -- 172.21.15.31:0/2669920432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000ccc0 con 0x7f1ec81036e0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.752+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a6d20 con 0x7f1ec81036e0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.752+0000 7f1ec4ff9640 1 -- 172.21.15.31:0/2669920432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb0010910 con 0x7f1ec81036e0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.752+0000 7f1ec4ff9640 1 -- 172.21.15.31:0/2669920432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0018920 con 0x7f1ec81036e0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.752+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a7250 con 0x7f1ec81036e0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.753+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec8103110 con 0x7f1ec81036e0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.754+0000 7f1ec4ff9640 1 -- 172.21.15.31:0/2669920432 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1eb0010ac0 con 0x7f1ec81036e0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.755+0000 7f1ec4ff9640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1eb40a32d0 0x7f1eb40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.755+0000 7f1ec4ff9640 1 -- 172.21.15.31:0/2669920432 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f1eb00d0550 con 0x7f1ec81036e0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.755+0000 7f1ec6ffd640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1eb40a32d0 0x7f1eb40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.755+0000 7f1ec6ffd640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1eb40a32d0 0x7f1eb40a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009bd0 tx=0x7f1eb8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.758+0000 7f1ec4ff9640 1 -- 172.21.15.31:0/2669920432 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1eb009cd80 con 0x7f1ec81036e0 2024-09-16T00:40:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:19 smithi102 ceph-mon[35364]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:19 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/660876079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:40:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.934+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1ec8101ec0 con 0x7f1ec81036e0 2024-09-16T00:40:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.935+0000 7f1ec4ff9640 1 -- 172.21.15.31:0/2669920432 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ec8101ec0 con 0x7f1ec81036e0 2024-09-16T00:40:19.935 INFO:teuthology.orchestra.run.smithi031.stdout:55834574930 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1eb40a32d0 msgr2=0x7f1eb40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1eb40a32d0 0x7f1eb40a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009bd0 tx=0x7f1eb8005f90 comp rx=0 tx=0).stop 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81036e0 msgr2=0x7f1ec8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81036e0 0x7f1ec8100ed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000e9f0 tx=0x7f1eb000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 shutdown_connections 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f1eb40a32d0 0x7f1eb40a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81082a0 0x7f1ec8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ec81036e0 0x7f1ec8100ed0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 --2- 172.21.15.31:0/2669920432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ec8102d30 0x7f1ec8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 >> 172.21.15.31:0/2669920432 conn(0x7f1ec80fc7d0 msgr2=0x7f1ec8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 shutdown_connections 2024-09-16T00:40:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:19.937+0000 7f1ecd434640 1 -- 172.21.15.31:0/2669920432 wait complete. 2024-09-16T00:40:20.230 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790010 got 158913790012 for osd.5 2024-09-16T00:40:20.230 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:20.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:20 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2669920432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:40:20.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 -- 172.21.15.31:0/639213398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441082a0 msgr2=0x7ff54410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/639213398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441082a0 0x7ff54410ec40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009cd0 tx=0x7ff53c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 -- 172.21.15.31:0/639213398 shutdown_connections 2024-09-16T00:40:20.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/639213398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441082a0 0x7ff54410ec40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/639213398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5441036e0 0x7ff544107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/639213398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff544102d30 0x7ff544103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 -- 172.21.15.31:0/639213398 >> 172.21.15.31:0/639213398 conn(0x7ff5440fc7d0 msgr2=0x7ff5440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 -- 172.21.15.31:0/639213398 shutdown_connections 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 -- 172.21.15.31:0/639213398 wait complete. 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.495+0000 7ff54a6ad640 1 Processor -- start 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff54a6ad640 1 -- start start 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff54a6ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff544102d30 0x7ff544198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff54a6ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441036e0 0x7ff544198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff54a6ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5441082a0 0x7ff54419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff54a6ad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441120a0 con 0x7ff5441036e0 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff54a6ad640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff544111f20 con 0x7ff544102d30 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff54a6ad640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff544112220 con 0x7ff5441082a0 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff5437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441036e0 0x7ff544198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff5437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441036e0 0x7ff544198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47180/0 (socket says 172.21.15.31:47180) 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff5437fe640 1 -- 172.21.15.31:0/1699876523 learned_addr learned my addr 172.21.15.31:0/1699876523 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff543fff640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff544102d30 0x7ff544198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.496+0000 7ff548c4a640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5441082a0 0x7ff54419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff5437fe640 1 -- 172.21.15.31:0/1699876523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5441082a0 msgr2=0x7ff54419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff5437fe640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5441082a0 0x7ff54419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff5437fe640 1 -- 172.21.15.31:0/1699876523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff544102d30 msgr2=0x7ff544198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff5437fe640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff544102d30 0x7ff544198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff5437fe640 1 -- 172.21.15.31:0/1699876523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54419d520 con 0x7ff5441036e0 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff548c4a640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5441082a0 0x7ff54419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff5437fe640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441036e0 0x7ff544198ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff53800d950 tx=0x7ff53800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff5417fa640 1 -- 172.21.15.31:0/1699876523 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538017070 con 0x7ff5441036e0 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54419d810 con 0x7ff5441036e0 2024-09-16T00:40:20.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff5417fa640 1 -- 172.21.15.31:0/1699876523 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff538009c30 con 0x7ff5441036e0 2024-09-16T00:40:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff5417fa640 1 -- 172.21.15.31:0/1699876523 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5380047a0 con 0x7ff5441036e0 2024-09-16T00:40:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.497+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5441a50f0 con 0x7ff5441036e0 2024-09-16T00:40:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.498+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff544103110 con 0x7ff5441036e0 2024-09-16T00:40:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.499+0000 7ff5417fa640 1 -- 172.21.15.31:0/1699876523 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff538004020 con 0x7ff5441036e0 2024-09-16T00:40:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.500+0000 7ff5417fa640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff52c0a32d0 0x7ff52c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.500+0000 7ff543fff640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff52c0a32d0 0x7ff52c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.500+0000 7ff5417fa640 1 -- 172.21.15.31:0/1699876523 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7ff5380cfa10 con 0x7ff5441036e0 2024-09-16T00:40:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.500+0000 7ff543fff640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff52c0a32d0 0x7ff52c0a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff534009bd0 tx=0x7ff534005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.504+0000 7ff5417fa640 1 -- 172.21.15.31:0/1699876523 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff538020020 con 0x7ff5441036e0 2024-09-16T00:40:20.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.688+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff54410a2e0 con 0x7ff5441036e0 2024-09-16T00:40:20.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.688+0000 7ff5417fa640 1 -- 172.21.15.31:0/1699876523 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff53809c240 con 0x7ff5441036e0 2024-09-16T00:40:20.689 INFO:teuthology.orchestra.run.smithi031.stdout:322122547231 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff52c0a32d0 msgr2=0x7ff52c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff52c0a32d0 0x7ff52c0a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff534009bd0 tx=0x7ff534005f90 comp rx=0 tx=0).stop 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441036e0 msgr2=0x7ff544198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441036e0 0x7ff544198ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff53800d950 tx=0x7ff53800de10 comp rx=0 tx=0).stop 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 shutdown_connections 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff52c0a32d0 0x7ff52c0a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5441082a0 0x7ff54419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5441036e0 0x7ff544198ab0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 --2- 172.21.15.31:0/1699876523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff544102d30 0x7ff544198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 >> 172.21.15.31:0/1699876523 conn(0x7ff5440fc7d0 msgr2=0x7ff544105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 shutdown_connections 2024-09-16T00:40:20.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:20.690+0000 7ff54a6ad640 1 -- 172.21.15.31:0/1699876523 wait complete. 2024-09-16T00:40:20.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:20 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2669920432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:40:20.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:20 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2669920432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:40:21.407 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561079 got 188978561080 for osd.6 2024-09-16T00:40:21.407 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:21.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:21 smithi031 ceph-mon[30956]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:21.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:21 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1699876523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:40:21.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:21 smithi081 ceph-mon[35539]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:21.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:21 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1699876523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:40:21.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:21 smithi102 ceph-mon[35364]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:21.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:21 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1699876523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 -- 172.21.15.31:0/3226453187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 msgr2=0x7f0b0810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 --2- 172.21.15.31:0/3226453187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 0x7f0b0810ec40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009cd0 tx=0x7f0b0002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 -- 172.21.15.31:0/3226453187 shutdown_connections 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 --2- 172.21.15.31:0/3226453187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 0x7f0b0810ec40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 --2- 172.21.15.31:0/3226453187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b081036e0 0x7f0b08107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 --2- 172.21.15.31:0/3226453187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b08102d30 0x7f0b08103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 -- 172.21.15.31:0/3226453187 >> 172.21.15.31:0/3226453187 conn(0x7f0b080fc7d0 msgr2=0x7f0b080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 -- 172.21.15.31:0/3226453187 shutdown_connections 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 -- 172.21.15.31:0/3226453187 wait complete. 2024-09-16T00:40:21.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.799+0000 7f0b0d410640 1 Processor -- start 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b0d410640 1 -- start start 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b0d410640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b08102d30 0x7f0b08198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b0d410640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b081036e0 0x7f0b08198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b0d410640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 0x7f0b0819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b0d410640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b081120a0 con 0x7f0b081082a0 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b0d410640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08111f20 con 0x7f0b081036e0 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b0d410640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08112220 con 0x7f0b08102d30 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b077fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 0x7f0b0819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0afe7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b081036e0 0x7f0b08198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b077fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 0x7f0b0819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47210/0 (socket says 172.21.15.31:47210) 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b077fe640 1 -- 172.21.15.31:0/1285630286 learned_addr learned my addr 172.21.15.31:0/1285630286 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b06ffd640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b08102d30 0x7f0b08198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b077fe640 1 -- 172.21.15.31:0/1285630286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b08102d30 msgr2=0x7f0b08198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b077fe640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b08102d30 0x7f0b08198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b077fe640 1 -- 172.21.15.31:0/1285630286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b081036e0 msgr2=0x7f0b08198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b077fe640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b081036e0 0x7f0b08198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.800+0000 7f0b077fe640 1 -- 172.21.15.31:0/1285630286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0819d500 con 0x7f0b081082a0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.801+0000 7f0b077fe640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 0x7f0b0819ce20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005cc0 tx=0x7f0b00003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.801+0000 7f0b04ff9640 1 -- 172.21.15.31:0/1285630286 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00004040 con 0x7f0b081082a0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.801+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0819d790 con 0x7f0b081082a0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.801+0000 7f0b04ff9640 1 -- 172.21.15.31:0/1285630286 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b000046c0 con 0x7f0b081082a0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.801+0000 7f0b04ff9640 1 -- 172.21.15.31:0/1285630286 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00046480 con 0x7f0b081082a0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.801+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b081a50d0 con 0x7f0b081082a0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.802+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b08103110 con 0x7f0b081082a0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.803+0000 7f0b04ff9640 1 -- 172.21.15.31:0/1285630286 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0b000305a0 con 0x7f0b081082a0 2024-09-16T00:40:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.803+0000 7f0b04ff9640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0af40a32d0 0x7f0af40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:21.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.804+0000 7f0b06ffd640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0af40a32d0 0x7f0af40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:21.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.804+0000 7f0b04ff9640 1 -- 172.21.15.31:0/1285630286 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f0b0002e030 con 0x7f0b081082a0 2024-09-16T00:40:21.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.804+0000 7f0b06ffd640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0af40a32d0 0x7f0af40a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0af800bbe0 tx=0x7f0af800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:21.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.808+0000 7f0b04ff9640 1 -- 172.21.15.31:0/1285630286 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b0004b060 con 0x7f0b081082a0 2024-09-16T00:40:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.985+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0b0810a2b0 con 0x7f0b081082a0 2024-09-16T00:40:21.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.985+0000 7f0b04ff9640 1 -- 172.21.15.31:0/1285630286 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0b000bdde0 con 0x7f0b081082a0 2024-09-16T00:40:21.986 INFO:teuthology.orchestra.run.smithi031.stdout:103079215175 2024-09-16T00:40:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.987+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0af40a32d0 msgr2=0x7f0af40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.987+0000 7f0b0d410640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0af40a32d0 0x7f0af40a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0af800bbe0 tx=0x7f0af800b390 comp rx=0 tx=0).stop 2024-09-16T00:40:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.987+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 msgr2=0x7f0b0819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.987+0000 7f0b0d410640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 0x7f0b0819ce20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005cc0 tx=0x7f0b00003970 comp rx=0 tx=0).stop 2024-09-16T00:40:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.988+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 shutdown_connections 2024-09-16T00:40:21.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.988+0000 7f0b0d410640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b081082a0 0x7f0b0819ce20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.988+0000 7f0b0d410640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b081036e0 0x7f0b08198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.988+0000 7f0b0d410640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0af40a32d0 0x7f0af40a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.988+0000 7f0b0d410640 1 --2- 172.21.15.31:0/1285630286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b08102d30 0x7f0b08198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:21.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.988+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 >> 172.21.15.31:0/1285630286 conn(0x7f0b080fc7d0 msgr2=0x7f0b08106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:21.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.988+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 shutdown_connections 2024-09-16T00:40:21.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:21.988+0000 7f0b0d410640 1 -- 172.21.15.31:0/1285630286 wait complete. 2024-09-16T00:40:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.029+0000 7fb843407640 1 -- 172.21.15.31:0/2485304335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 msgr2=0x7fb83c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.029+0000 7fb843407640 1 --2- 172.21.15.31:0/2485304335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 0x7fb83c10eaa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb834009ca0 tx=0x7fb83402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.029+0000 7fb843407640 1 -- 172.21.15.31:0/2485304335 shutdown_connections 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.029+0000 7fb843407640 1 --2- 172.21.15.31:0/2485304335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 0x7fb83c10eaa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.029+0000 7fb843407640 1 --2- 172.21.15.31:0/2485304335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb83c1027a0 0x7fb83c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.029+0000 7fb843407640 1 --2- 172.21.15.31:0/2485304335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb83c101df0 0x7fb83c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.029+0000 7fb843407640 1 -- 172.21.15.31:0/2485304335 >> 172.21.15.31:0/2485304335 conn(0x7fb83c0fd8f0 msgr2=0x7fb83c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.029+0000 7fb843407640 1 -- 172.21.15.31:0/2485304335 shutdown_connections 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.029+0000 7fb843407640 1 -- 172.21.15.31:0/2485304335 wait complete. 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb843407640 1 Processor -- start 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb843407640 1 -- start start 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb843407640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb83c101df0 0x7fb83c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb843407640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb83c1027a0 0x7fb83c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb843407640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 0x7fb83c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb843407640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c111ff0 con 0x7fb83c106f20 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb843407640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c111e70 con 0x7fb83c1027a0 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb843407640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c112170 con 0x7fb83c101df0 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb8419a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 0x7fb83c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb8419a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 0x7fb83c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47238/0 (socket says 172.21.15.31:47238) 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.030+0000 7fb8419a4640 1 -- 172.21.15.31:0/770444062 learned_addr learned my addr 172.21.15.31:0/770444062 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb8409a2640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb83c1027a0 0x7fb83c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb8411a3640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb83c101df0 0x7fb83c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb8419a4640 1 -- 172.21.15.31:0/770444062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb83c101df0 msgr2=0x7fb83c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb8419a4640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb83c101df0 0x7fb83c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb8419a4640 1 -- 172.21.15.31:0/770444062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb83c1027a0 msgr2=0x7fb83c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb8419a4640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb83c1027a0 0x7fb83c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb8419a4640 1 -- 172.21.15.31:0/770444062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c19d3c0 con 0x7fb83c106f20 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb8409a2640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb83c1027a0 0x7fb83c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb8419a4640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 0x7fb83c19cce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb834033040 tx=0x7fb834003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb82e7fc640 1 -- 172.21.15.31:0/770444062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834031650 con 0x7fb83c106f20 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb82e7fc640 1 -- 172.21.15.31:0/770444062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8340317f0 con 0x7fb83c106f20 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb82e7fc640 1 -- 172.21.15.31:0/770444062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83402f980 con 0x7fb83c106f20 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83c19d650 con 0x7fb83c106f20 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.031+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83c1a4f90 con 0x7fb83c106f20 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.032+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb83c1021d0 con 0x7fb83c106f20 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.034+0000 7fb82e7fc640 1 -- 172.21.15.31:0/770444062 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb834004b40 con 0x7fb83c106f20 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.035+0000 7fb82e7fc640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8280a3350 0x7fb8280a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.035+0000 7fb8411a3640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8280a3350 0x7fb8280a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.035+0000 7fb82e7fc640 1 -- 172.21.15.31:0/770444062 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fb8340f12c0 con 0x7fb83c106f20 2024-09-16T00:40:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.036+0000 7fb8411a3640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8280a3350 0x7fb8280a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb838009990 tx=0x7fb838009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.038+0000 7fb82e7fc640 1 -- 172.21.15.31:0/770444062 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8340bdc00 con 0x7fb83c106f20 2024-09-16T00:40:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.214+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb83c199770 con 0x7fb83c106f20 2024-09-16T00:40:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.215+0000 7fb82e7fc640 1 -- 172.21.15.31:0/770444062 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb83c199770 con 0x7fb83c106f20 2024-09-16T00:40:22.215 INFO:teuthology.orchestra.run.smithi031.stdout:34359738455 2024-09-16T00:40:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8280a3350 msgr2=0x7fb8280a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8280a3350 0x7fb8280a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb838009990 tx=0x7fb838009110 comp rx=0 tx=0).stop 2024-09-16T00:40:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 msgr2=0x7fb83c19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 0x7fb83c19cce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb834033040 tx=0x7fb834003f10 comp rx=0 tx=0).stop 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 shutdown_connections 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83c106f20 0x7fb83c19cce0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb83c1027a0 0x7fb83c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8280a3350 0x7fb8280a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 --2- 172.21.15.31:0/770444062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb83c101df0 0x7fb83c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 >> 172.21.15.31:0/770444062 conn(0x7fb83c0fd8f0 msgr2=0x7fb83c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 shutdown_connections 2024-09-16T00:40:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:22.217+0000 7fb843407640 1 -- 172.21.15.31:0/770444062 wait complete. 2024-09-16T00:40:22.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:22 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1285630286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:40:22.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:22 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/770444062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:40:22.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:22 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1285630286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:40:22.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:22 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/770444062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:40:22.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:22 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1285630286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:40:22.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:22 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/770444062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:40:23.120 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574930 for osd.1 2024-09-16T00:40:23.120 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:23.608 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444107 got 73014444109 for osd.2 2024-09-16T00:40:23.608 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:23.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:23 smithi031 ceph-mon[30956]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:23 smithi081 ceph-mon[35539]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:23.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:23 smithi102 ceph-mon[35364]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:24.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.223+0000 7f20ea8b9640 1 -- 172.21.15.31:0/220743172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4100a80 msgr2=0x7f20e4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:24.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.223+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/220743172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4100a80 0x7f20e4100ee0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009930 tx=0x7f20d802a420 comp rx=0 tx=0).stop 2024-09-16T00:40:24.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.223+0000 7f20ea8b9640 1 -- 172.21.15.31:0/220743172 shutdown_connections 2024-09-16T00:40:24.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.223+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/220743172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20e4101420 0x7f20e410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.223+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/220743172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4100a80 0x7f20e4100ee0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.223+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/220743172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e4106a80 0x7f20e4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.223+0000 7f20ea8b9640 1 -- 172.21.15.31:0/220743172 >> 172.21.15.31:0/220743172 conn(0x7f20e40fc7a0 msgr2=0x7f20e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.223+0000 7f20ea8b9640 1 -- 172.21.15.31:0/220743172 shutdown_connections 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.223+0000 7f20ea8b9640 1 -- 172.21.15.31:0/220743172 wait complete. 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20ea8b9640 1 Processor -- start 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20ea8b9640 1 -- start start 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20ea8b9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e4100a80 0x7f20e4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20ea8b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20e4101420 0x7f20e4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20ea8b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4106a80 0x7f20e419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20ea8b9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4110e80 con 0x7f20e4106a80 2024-09-16T00:40:24.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20ea8b9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4110d00 con 0x7f20e4101420 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20ea8b9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4111000 con 0x7f20e4100a80 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20e8e56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4106a80 0x7f20e419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20e8e56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4106a80 0x7f20e419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47268/0 (socket says 172.21.15.31:47268) 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20e8e56640 1 -- 172.21.15.31:0/49261404 learned_addr learned my addr 172.21.15.31:0/49261404 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20e37fe640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20e4101420 0x7f20e4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.224+0000 7f20e3fff640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e4100a80 0x7f20e4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20e8e56640 1 -- 172.21.15.31:0/49261404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e4100a80 msgr2=0x7f20e4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20e8e56640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e4100a80 0x7f20e4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20e8e56640 1 -- 172.21.15.31:0/49261404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20e4101420 msgr2=0x7f20e4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20e8e56640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20e4101420 0x7f20e4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20e8e56640 1 -- 172.21.15.31:0/49261404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e419d400 con 0x7f20e4106a80 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20e8e56640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4106a80 0x7f20e419cd20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00eaa0 tx=0x7f20dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:24.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20e17fa640 1 -- 172.21.15.31:0/49261404 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00cdb0 con 0x7f20e4106a80 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20e17fa640 1 -- 172.21.15.31:0/49261404 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20dc005020 con 0x7f20e4106a80 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e419d6f0 con 0x7f20e4106a80 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20e17fa640 1 -- 172.21.15.31:0/49261404 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0116b0 con 0x7f20e4106a80 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.225+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e41a4fd0 con 0x7f20e4106a80 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.226+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e4100ee0 con 0x7f20e4106a80 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.228+0000 7f20e17fa640 1 -- 172.21.15.31:0/49261404 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f20dc005560 con 0x7f20e4106a80 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.229+0000 7f20e17fa640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20cc0a32d0 0x7f20cc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.229+0000 7f20e3fff640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20cc0a32d0 0x7f20cc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.229+0000 7f20e17fa640 1 -- 172.21.15.31:0/49261404 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f20dc0d02e0 con 0x7f20e4106a80 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.229+0000 7f20e3fff640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20cc0a32d0 0x7f20cc0a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20d40058c0 tx=0x7f20d400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.232+0000 7f20e17fa640 1 -- 172.21.15.31:0/49261404 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20dc016030 con 0x7f20e4106a80 2024-09-16T00:40:24.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.411+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f20e41997b0 con 0x7f20e4106a80 2024-09-16T00:40:24.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.412+0000 7f20e17fa640 1 -- 172.21.15.31:0/49261404 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f20e41997b0 con 0x7f20e4106a80 2024-09-16T00:40:24.412 INFO:teuthology.orchestra.run.smithi031.stdout:137438953539 2024-09-16T00:40:24.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20cc0a32d0 msgr2=0x7f20cc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:24.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20cc0a32d0 0x7f20cc0a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20d40058c0 tx=0x7f20d400a250 comp rx=0 tx=0).stop 2024-09-16T00:40:24.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4106a80 msgr2=0x7f20e419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:24.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4106a80 0x7f20e419cd20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00eaa0 tx=0x7f20dc00ef60 comp rx=0 tx=0).stop 2024-09-16T00:40:24.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 shutdown_connections 2024-09-16T00:40:24.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e4106a80 0x7f20e419cd20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:24.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20e4101420 0x7f20e4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:24.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f20cc0a32d0 0x7f20cc0a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:24.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 --2- 172.21.15.31:0/49261404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20e4100a80 0x7f20e4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:24.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 >> 172.21.15.31:0/49261404 conn(0x7f20e40fc7a0 msgr2=0x7f20e4104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:24.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 shutdown_connections 2024-09-16T00:40:24.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:24.414+0000 7f20ea8b9640 1 -- 172.21.15.31:0/49261404 wait complete. 2024-09-16T00:40:24.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:24 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/49261404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:40:24.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:24 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/49261404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:40:24.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:24 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/49261404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:40:24.801 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738455 for osd.0 2024-09-16T00:40:24.801 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:25.572 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953536 got 137438953539 for osd.4 2024-09-16T00:40:25.573 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:25.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:25 smithi081 ceph-mon[35539]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:25.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:25 smithi102 ceph-mon[35364]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:25.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:25 smithi031 ceph-mon[30956]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:26.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397552 got 210453397555 for osd.7 2024-09-16T00:40:26.033 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:26.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.259+0000 7fe9d99b5640 1 -- 172.21.15.31:0/895254193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 msgr2=0x7fe9d410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.259+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/895254193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 0x7fe9d410d950 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009cd0 tx=0x7fe9cc02a8f0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.259+0000 7fe9d99b5640 1 -- 172.21.15.31:0/895254193 shutdown_connections 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.259+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/895254193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 0x7fe9d410d950 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.259+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/895254193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d4100a80 0x7fe9d4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.259+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/895254193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9d4106a80 0x7fe9d4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.259+0000 7fe9d99b5640 1 -- 172.21.15.31:0/895254193 >> 172.21.15.31:0/895254193 conn(0x7fe9d40fc7a0 msgr2=0x7fe9d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.259+0000 7fe9d99b5640 1 -- 172.21.15.31:0/895254193 shutdown_connections 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.259+0000 7fe9d99b5640 1 -- 172.21.15.31:0/895254193 wait complete. 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.260+0000 7fe9d99b5640 1 Processor -- start 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.260+0000 7fe9d99b5640 1 -- start start 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.260+0000 7fe9d99b5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d4100a80 0x7fe9d419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.260+0000 7fe9d99b5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 0x7fe9d419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.260+0000 7fe9d99b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9d4106a80 0x7fe9d419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.260+0000 7fe9d99b5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4110e80 con 0x7fe9d4101420 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.260+0000 7fe9d99b5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4110d00 con 0x7fe9d4106a80 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.260+0000 7fe9d99b5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4111000 con 0x7fe9d4100a80 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.260+0000 7fe9d27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 0x7fe9d419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 0x7fe9d419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47284/0 (socket says 172.21.15.31:47284) 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d27fc640 1 -- 172.21.15.31:0/3037803586 learned_addr learned my addr 172.21.15.31:0/3037803586 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d37fe640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9d4106a80 0x7fe9d419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d2ffd640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d4100a80 0x7fe9d419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d27fc640 1 -- 172.21.15.31:0/3037803586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d4100a80 msgr2=0x7fe9d419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d27fc640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d4100a80 0x7fe9d419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d27fc640 1 -- 172.21.15.31:0/3037803586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9d4106a80 msgr2=0x7fe9d419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:26.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d27fc640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9d4106a80 0x7fe9d419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d27fc640 1 -- 172.21.15.31:0/3037803586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d419f500 con 0x7fe9d4101420 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d27fc640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 0x7fe9d419aa90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800c920 tx=0x7fe9c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d89b3640 1 -- 172.21.15.31:0/3037803586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8007c70 con 0x7fe9d4101420 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d89b3640 1 -- 172.21.15.31:0/3037803586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c8007e10 con 0x7fe9d4101420 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d419f7f0 con 0x7fe9d4101420 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.261+0000 7fe9d89b3640 1 -- 172.21.15.31:0/3037803586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8018890 con 0x7fe9d4101420 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.262+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d41a70d0 con 0x7fe9d4101420 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.263+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d4100ee0 con 0x7fe9d4101420 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.264+0000 7fe9d89b3640 1 -- 172.21.15.31:0/3037803586 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe9c8018a50 con 0x7fe9d4101420 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.265+0000 7fe9d89b3640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9bc0a3350 0x7fe9bc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.265+0000 7fe9d2ffd640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9bc0a3350 0x7fe9bc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.265+0000 7fe9d89b3640 1 -- 172.21.15.31:0/3037803586 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fe9c80d0150 con 0x7fe9d4101420 2024-09-16T00:40:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.265+0000 7fe9d2ffd640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9bc0a3350 0x7fe9bc0a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400cac0 tx=0x7fe9c400c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.269+0000 7fe9d89b3640 1 -- 172.21.15.31:0/3037803586 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9c8014030 con 0x7fe9d4101420 2024-09-16T00:40:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.451+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe9d419b8b0 con 0x7fe9d4101420 2024-09-16T00:40:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.451+0000 7fe9d89b3640 1 -- 172.21.15.31:0/3037803586 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe9c809caf0 con 0x7fe9d4101420 2024-09-16T00:40:26.452 INFO:teuthology.orchestra.run.smithi031.stdout:240518168624 2024-09-16T00:40:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.454+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9bc0a3350 msgr2=0x7fe9bc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.454+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9bc0a3350 0x7fe9bc0a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400cac0 tx=0x7fe9c400c4a0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.454+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 msgr2=0x7fe9d419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.454+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 0x7fe9d419aa90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800c920 tx=0x7fe9c800cde0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.454+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 shutdown_connections 2024-09-16T00:40:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.454+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9d4106a80 0x7fe9d419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.454+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9d4101420 0x7fe9d419aa90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.454+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9bc0a3350 0x7fe9bc0a5810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.454+0000 7fe9d99b5640 1 --2- 172.21.15.31:0/3037803586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d4100a80 0x7fe9d419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.455+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 >> 172.21.15.31:0/3037803586 conn(0x7fe9d40fc7a0 msgr2=0x7fe9d406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.455+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 shutdown_connections 2024-09-16T00:40:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:26.455+0000 7fe9d99b5640 1 -- 172.21.15.31:0/3037803586 wait complete. 2024-09-16T00:40:26.749 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776164 for osd.10 2024-09-16T00:40:26.750 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:27.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547229 got 322122547231 for osd.11 2024-09-16T00:40:27.352 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:27.595 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:27 smithi031 ceph-mon[30956]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:27.595 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:27 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3037803586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:40:27.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:27 smithi081 ceph-mon[35539]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:27.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:27 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3037803586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:40:27.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:27 smithi102 ceph-mon[35364]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:27.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:27 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3037803586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:40:28.013 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168620 got 240518168624 for osd.8 2024-09-16T00:40:28.014 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:28.443 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215171 got 103079215175 for osd.3 2024-09-16T00:40:28.444 DEBUG:teuthology.parallel:result is None 2024-09-16T00:40:28.444 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:40:28.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:40:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 -- 172.21.15.31:0/363023941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 msgr2=0x7f87e01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 --2- 172.21.15.31:0/363023941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 0x7f87e01031a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009900 tx=0x7f87d002a480 comp rx=0 tx=0).stop 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 -- 172.21.15.31:0/363023941 shutdown_connections 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 --2- 172.21.15.31:0/363023941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87e010c160 0x7f87e010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 --2- 172.21.15.31:0/363023941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87e01036e0 0x7f87e010bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 --2- 172.21.15.31:0/363023941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 0x7f87e01031a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 -- 172.21.15.31:0/363023941 >> 172.21.15.31:0/363023941 conn(0x7f87e00fc780 msgr2=0x7f87e00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 -- 172.21.15.31:0/363023941 shutdown_connections 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 -- 172.21.15.31:0/363023941 wait complete. 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.211+0000 7f87e51dc640 1 Processor -- start 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87e51dc640 1 -- start start 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87e51dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 0x7f87e0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:29.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87e51dc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87e01036e0 0x7f87e01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87e51dc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87e010c160 0x7f87e019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87e51dc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0111f50 con 0x7f87e0102dc0 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87e51dc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0111dd0 con 0x7f87e010c160 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87e51dc640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01120d0 con 0x7f87e01036e0 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87ded9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 0x7f87e0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87ded9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 0x7f87e0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49806/0 (socket says 172.21.15.31:49806) 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87ded9d640 1 -- 172.21.15.31:0/843113018 learned_addr learned my addr 172.21.15.31:0/843113018 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87de59c640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87e01036e0 0x7f87e01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87df59e640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87e010c160 0x7f87e019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87ded9d640 1 -- 172.21.15.31:0/843113018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87e01036e0 msgr2=0x7f87e01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87ded9d640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87e01036e0 0x7f87e01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87ded9d640 1 -- 172.21.15.31:0/843113018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87e010c160 msgr2=0x7f87e019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.212+0000 7f87ded9d640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87e010c160 0x7f87e019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.213+0000 7f87ded9d640 1 -- 172.21.15.31:0/843113018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e019d420 con 0x7f87e0102dc0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.213+0000 7f87ded9d640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 0x7f87e0198470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f87d0006010 tx=0x7f87d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.213+0000 7f87c7fff640 1 -- 172.21.15.31:0/843113018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d003f030 con 0x7f87e0102dc0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.213+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e019d6b0 con 0x7f87e0102dc0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.213+0000 7f87c7fff640 1 -- 172.21.15.31:0/843113018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d0003ce0 con 0x7f87e0102dc0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.213+0000 7f87c7fff640 1 -- 172.21.15.31:0/843113018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0035070 con 0x7f87e0102dc0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.213+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e01a4ff0 con 0x7f87e0102dc0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.214+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e01031a0 con 0x7f87e0102dc0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.215+0000 7f87c7fff640 1 -- 172.21.15.31:0/843113018 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f87d0042030 con 0x7f87e0102dc0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.216+0000 7f87c7fff640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f87c80a3350 0x7f87c80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.216+0000 7f87c7fff640 1 -- 172.21.15.31:0/843113018 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f87d00f0240 con 0x7f87e0102dc0 2024-09-16T00:40:29.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.216+0000 7f87de59c640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f87c80a3350 0x7f87c80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.216+0000 7f87de59c640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f87c80a3350 0x7f87c80a5810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f87e00fdc20 tx=0x7f87d400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:29.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.220+0000 7f87c7fff640 1 -- 172.21.15.31:0/843113018 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87d00bc4e0 con 0x7f87e0102dc0 2024-09-16T00:40:29.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.394+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87e0062630 con 0x7f87c80a3350 2024-09-16T00:40:29.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.402+0000 7f87c7fff640 1 -- 172.21.15.31:0/843113018 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340285 (secure 0 0 0) 0x7f87e0062630 con 0x7f87c80a3350 2024-09-16T00:40:29.407 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:29.409 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:40:29.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f87c80a3350 msgr2=0x7f87c80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:29.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f87c80a3350 0x7f87c80a5810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f87e00fdc20 tx=0x7f87d400b390 comp rx=0 tx=0).stop 2024-09-16T00:40:29.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 msgr2=0x7f87e0198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 0x7f87e0198470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f87d0006010 tx=0x7f87d00042f0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 shutdown_connections 2024-09-16T00:40:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87e010c160 0x7f87e019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f87c80a3350 0x7f87c80a5810 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87e01036e0 0x7f87e01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 --2- 172.21.15.31:0/843113018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0102dc0 0x7f87e0198470 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 >> 172.21.15.31:0/843113018 conn(0x7f87e00fc780 msgr2=0x7f87e00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 shutdown_connections 2024-09-16T00:40:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:29.406+0000 7f87e51dc640 1 -- 172.21.15.31:0/843113018 wait complete. 2024-09-16T00:40:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:29 smithi081 ceph-mon[35539]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:29.780 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:29 smithi031 ceph-mon[30956]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:29 smithi102 ceph-mon[35364]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:30.099 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-16T00:40:27.675496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654911,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":243,"num_read_kb":232,"num_write":268,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400524,"kb_used_data":13760,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541300,"statfs":{"total":1151940427776,"available":1151530291200,"internally_reserved":0,"allocated":14090240,"data_stored":7941443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002966"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373754+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:39:59.373754+0000","last_peered":"2024-09-16T00:39:59.373754+0000","last_clean":"2024-09-16T00:39:59.373754+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:39:59.373754+0000","last_undegraded":"2024-09-16T00:39:59.373754+0000","last_fullsized":"2024-09-16T00:39:59.373754+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383882+0000","last_change":"2024-09-16T00:38:52.968681+0000","last_active":"2024-09-16T00:39:59.383882+0000","last_peered":"2024-09-16T00:39:59.383882+0000","last_clean":"2024-09-16T00:39:59.383882+0000","last_became_active":"2024-09-16T00:38:52.968359+0000","last_became_peered":"2024-09-16T00:38:52.968359+0000","last_unstale":"2024-09-16T00:39:59.383882+0000","last_undegraded":"2024-09-16T00:39:59.383882+0000","last_fullsized":"2024-09-16T00:39:59.383882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:17.870176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.380513+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:39:59.380513+0000","last_peered":"2024-09-16T00:39:59.380513+0000","last_clean":"2024-09-16T00:39:59.380513+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:39:59.380513+0000","last_undegraded":"2024-09-16T00:39:59.380513+0000","last_fullsized":"2024-09-16T00:39:59.380513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.384005+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:39:59.384005+0000","last_peered":"2024-09-16T00:39:59.384005+0000","last_clean":"2024-09-16T00:39:59.384005+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:39:59.384005+0000","last_undegraded":"2024-09-16T00:39:59.384005+0000","last_fullsized":"2024-09-16T00:39:59.384005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.591997+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:39:59.591997+0000","last_peered":"2024-09-16T00:39:59.591997+0000","last_clean":"2024-09-16T00:39:59.591997+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:39:59.591997+0000","last_undegraded":"2024-09-16T00:39:59.591997+0000","last_fullsized":"2024-09-16T00:39:59.591997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.372931+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:39:59.372931+0000","last_peered":"2024-09-16T00:39:59.372931+0000","last_clean":"2024-09-16T00:39:59.372931+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:39:59.372931+0000","last_undegraded":"2024-09-16T00:39:59.372931+0000","last_fullsized":"2024-09-16T00:39:59.372931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374941+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:39:59.374941+0000","last_peered":"2024-09-16T00:39:59.374941+0000","last_clean":"2024-09-16T00:39:59.374941+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:39:59.374941+0000","last_undegraded":"2024-09-16T00:39:59.374941+0000","last_fullsized":"2024-09-16T00:39:59.374941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589523+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:39:59.589523+0000","last_peered":"2024-09-16T00:39:59.589523+0000","last_clean":"2024-09-16T00:39:59.589523+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:39:59.589523+0000","last_undegraded":"2024-09-16T00:39:59.589523+0000","last_fullsized":"2024-09-16T00:39:59.589523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373244+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:39:59.373244+0000","last_peered":"2024-09-16T00:39:59.373244+0000","last_clean":"2024-09-16T00:39:59.373244+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:39:59.373244+0000","last_undegraded":"2024-09-16T00:39:59.373244+0000","last_fullsized":"2024-09-16T00:39:59.373244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374246+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:39:59.374246+0000","last_peered":"2024-09-16T00:39:59.374246+0000","last_clean":"2024-09-16T00:39:59.374246+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:39:59.374246+0000","last_undegraded":"2024-09-16T00:39:59.374246+0000","last_fullsized":"2024-09-16T00:39:59.374246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371819+0000","last_change":"2024-09-16T00:38:52.975634+0000","last_active":"2024-09-16T00:39:59.371819+0000","last_peered":"2024-09-16T00:39:59.371819+0000","last_clean":"2024-09-16T00:39:59.371819+0000","last_became_active":"2024-09-16T00:38:52.975314+0000","last_became_peered":"2024-09-16T00:38:52.975314+0000","last_unstale":"2024-09-16T00:39:59.371819+0000","last_undegraded":"2024-09-16T00:39:59.371819+0000","last_fullsized":"2024-09-16T00:39:59.371819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:28.716600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373159+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:39:59.373159+0000","last_peered":"2024-09-16T00:39:59.373159+0000","last_clean":"2024-09-16T00:39:59.373159+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:39:59.373159+0000","last_undegraded":"2024-09-16T00:39:59.373159+0000","last_fullsized":"2024-09-16T00:39:59.373159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371178+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:39:59.371178+0000","last_peered":"2024-09-16T00:39:59.371178+0000","last_clean":"2024-09-16T00:39:59.371178+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:39:59.371178+0000","last_undegraded":"2024-09-16T00:39:59.371178+0000","last_fullsized":"2024-09-16T00:39:59.371178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373680+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:39:59.373680+0000","last_peered":"2024-09-16T00:39:59.373680+0000","last_clean":"2024-09-16T00:39:59.373680+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:39:59.373680+0000","last_undegraded":"2024-09-16T00:39:59.373680+0000","last_fullsized":"2024-09-16T00:39:59.373680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373320+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:39:59.373320+0000","last_peered":"2024-09-16T00:39:59.373320+0000","last_clean":"2024-09-16T00:39:59.373320+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:39:59.373320+0000","last_undegraded":"2024-09-16T00:39:59.373320+0000","last_fullsized":"2024-09-16T00:39:59.373320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381978+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:39:59.381978+0000","last_peered":"2024-09-16T00:39:59.381978+0000","last_clean":"2024-09-16T00:39:59.381978+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:39:59.381978+0000","last_undegraded":"2024-09-16T00:39:59.381978+0000","last_fullsized":"2024-09-16T00:39:59.381978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.591565+0000","last_change":"2024-09-16T00:39:59.591063+0000","last_active":"2024-09-16T00:39:59.591565+0000","last_peered":"2024-09-16T00:39:59.591565+0000","last_clean":"2024-09-16T00:39:59.591565+0000","last_became_active":"2024-09-16T00:39:59.590574+0000","last_became_peered":"2024-09-16T00:39:59.590574+0000","last_unstale":"2024-09-16T00:39:59.591565+0000","last_undegraded":"2024-09-16T00:39:59.591565+0000","last_fullsized":"2024-09-16T00:39:59.591565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:57.020592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.595772+0000","last_change":"2024-09-16T00:39:59.595772+0000","last_active":"2024-09-16T00:39:59.595772+0000","last_peered":"2024-09-16T00:39:59.595772+0000","last_clean":"2024-09-16T00:39:59.595772+0000","last_became_active":"2024-09-16T00:39:59.590772+0000","last_became_peered":"2024-09-16T00:39:59.590772+0000","last_unstale":"2024-09-16T00:39:59.595772+0000","last_undegraded":"2024-09-16T00:39:59.595772+0000","last_fullsized":"2024-09-16T00:39:59.595772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:53.236019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373961+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:39:59.373961+0000","last_peered":"2024-09-16T00:39:59.373961+0000","last_clean":"2024-09-16T00:39:59.373961+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:39:59.373961+0000","last_undegraded":"2024-09-16T00:39:59.373961+0000","last_fullsized":"2024-09-16T00:39:59.373961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.382933+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:39:59.382933+0000","last_peered":"2024-09-16T00:39:59.382933+0000","last_clean":"2024-09-16T00:39:59.382933+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:39:59.382933+0000","last_undegraded":"2024-09-16T00:39:59.382933+0000","last_fullsized":"2024-09-16T00:39:59.382933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.386827+0000","last_change":"2024-09-16T00:39:59.386827+0000","last_active":"2024-09-16T00:39:59.386827+0000","last_peered":"2024-09-16T00:39:59.386827+0000","last_clean":"2024-09-16T00:39:59.386827+0000","last_became_active":"2024-09-16T00:39:59.386389+0000","last_became_peered":"2024-09-16T00:39:59.386389+0000","last_unstale":"2024-09-16T00:39:59.386827+0000","last_undegraded":"2024-09-16T00:39:59.386827+0000","last_fullsized":"2024-09-16T00:39:59.386827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:12:21.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373450+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:39:59.373450+0000","last_peered":"2024-09-16T00:39:59.373450+0000","last_clean":"2024-09-16T00:39:59.373450+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:39:59.373450+0000","last_undegraded":"2024-09-16T00:39:59.373450+0000","last_fullsized":"2024-09-16T00:39:59.373450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589918+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:39:59.589918+0000","last_peered":"2024-09-16T00:39:59.589918+0000","last_clean":"2024-09-16T00:39:59.589918+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:39:59.589918+0000","last_undegraded":"2024-09-16T00:39:59.589918+0000","last_fullsized":"2024-09-16T00:39:59.589918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373345+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:39:59.373345+0000","last_peered":"2024-09-16T00:39:59.373345+0000","last_clean":"2024-09-16T00:39:59.373345+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:39:59.373345+0000","last_undegraded":"2024-09-16T00:39:59.373345+0000","last_fullsized":"2024-09-16T00:39:59.373345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590946+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:39:59.590946+0000","last_peered":"2024-09-16T00:39:59.590946+0000","last_clean":"2024-09-16T00:39:59.590946+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:39:59.590946+0000","last_undegraded":"2024-09-16T00:39:59.590946+0000","last_fullsized":"2024-09-16T00:39:59.590946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373627+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:39:59.373627+0000","last_peered":"2024-09-16T00:39:59.373627+0000","last_clean":"2024-09-16T00:39:59.373627+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:39:59.373627+0000","last_undegraded":"2024-09-16T00:39:59.373627+0000","last_fullsized":"2024-09-16T00:39:59.373627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.380670+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:39:59.380670+0000","last_peered":"2024-09-16T00:39:59.380670+0000","last_clean":"2024-09-16T00:39:59.380670+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:39:59.380670+0000","last_undegraded":"2024-09-16T00:39:59.380670+0000","last_fullsized":"2024-09-16T00:39:59.380670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383652+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:39:59.383652+0000","last_peered":"2024-09-16T00:39:59.383652+0000","last_clean":"2024-09-16T00:39:59.383652+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:39:59.383652+0000","last_undegraded":"2024-09-16T00:39:59.383652+0000","last_fullsized":"2024-09-16T00:39:59.383652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374360+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:39:59.374360+0000","last_peered":"2024-09-16T00:39:59.374360+0000","last_clean":"2024-09-16T00:39:59.374360+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:39:59.374360+0000","last_undegraded":"2024-09-16T00:39:59.374360+0000","last_fullsized":"2024-09-16T00:39:59.374360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381531+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:39:59.381531+0000","last_peered":"2024-09-16T00:39:59.381531+0000","last_clean":"2024-09-16T00:39:59.381531+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:39:59.381531+0000","last_undegraded":"2024-09-16T00:39:59.381531+0000","last_fullsized":"2024-09-16T00:39:59.381531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.378161+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:39:59.378161+0000","last_peered":"2024-09-16T00:39:59.378161+0000","last_clean":"2024-09-16T00:39:59.378161+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:39:59.378161+0000","last_undegraded":"2024-09-16T00:39:59.378161+0000","last_fullsized":"2024-09-16T00:39:59.378161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.375058+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:39:59.375058+0000","last_peered":"2024-09-16T00:39:59.375058+0000","last_clean":"2024-09-16T00:39:59.375058+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:39:59.375058+0000","last_undegraded":"2024-09-16T00:39:59.375058+0000","last_fullsized":"2024-09-16T00:39:59.375058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374082+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:39:59.374082+0000","last_peered":"2024-09-16T00:39:59.374082+0000","last_clean":"2024-09-16T00:39:59.374082+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:39:59.374082+0000","last_undegraded":"2024-09-16T00:39:59.374082+0000","last_fullsized":"2024-09-16T00:39:59.374082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589288+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:39:59.589288+0000","last_peered":"2024-09-16T00:39:59.589288+0000","last_clean":"2024-09-16T00:39:59.589288+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:39:59.589288+0000","last_undegraded":"2024-09-16T00:39:59.589288+0000","last_fullsized":"2024-09-16T00:39:59.589288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374366+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:39:59.374366+0000","last_peered":"2024-09-16T00:39:59.374366+0000","last_clean":"2024-09-16T00:39:59.374366+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:39:59.374366+0000","last_undegraded":"2024-09-16T00:39:59.374366+0000","last_fullsized":"2024-09-16T00:39:59.374366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373604+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:39:59.373604+0000","last_peered":"2024-09-16T00:39:59.373604+0000","last_clean":"2024-09-16T00:39:59.373604+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:39:59.373604+0000","last_undegraded":"2024-09-16T00:39:59.373604+0000","last_fullsized":"2024-09-16T00:39:59.373604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589506+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:39:59.589506+0000","last_peered":"2024-09-16T00:39:59.589506+0000","last_clean":"2024-09-16T00:39:59.589506+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:39:59.589506+0000","last_undegraded":"2024-09-16T00:39:59.589506+0000","last_fullsized":"2024-09-16T00:39:59.589506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371437+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:39:59.371437+0000","last_peered":"2024-09-16T00:39:59.371437+0000","last_clean":"2024-09-16T00:39:59.371437+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:39:59.371437+0000","last_undegraded":"2024-09-16T00:39:59.371437+0000","last_fullsized":"2024-09-16T00:39:59.371437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377209+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:39:59.377209+0000","last_peered":"2024-09-16T00:39:59.377209+0000","last_clean":"2024-09-16T00:39:59.377209+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:39:59.377209+0000","last_undegraded":"2024-09-16T00:39:59.377209+0000","last_fullsized":"2024-09-16T00:39:59.377209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371391+0000","last_change":"2024-09-16T00:38:52.972145+0000","last_active":"2024-09-16T00:39:59.371391+0000","last_peered":"2024-09-16T00:39:59.371391+0000","last_clean":"2024-09-16T00:39:59.371391+0000","last_became_active":"2024-09-16T00:38:52.971847+0000","last_became_peered":"2024-09-16T00:38:52.971847+0000","last_unstale":"2024-09-16T00:39:59.371391+0000","last_undegraded":"2024-09-16T00:39:59.371391+0000","last_fullsized":"2024-09-16T00:39:59.371391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:32.819389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377252+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:39:59.377252+0000","last_peered":"2024-09-16T00:39:59.377252+0000","last_clean":"2024-09-16T00:39:59.377252+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:39:59.377252+0000","last_undegraded":"2024-09-16T00:39:59.377252+0000","last_fullsized":"2024-09-16T00:39:59.377252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.378369+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:39:59.378369+0000","last_peered":"2024-09-16T00:39:59.378369+0000","last_clean":"2024-09-16T00:39:59.378369+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:39:59.378369+0000","last_undegraded":"2024-09-16T00:39:59.378369+0000","last_fullsized":"2024-09-16T00:39:59.378369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377051+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:39:59.377051+0000","last_peered":"2024-09-16T00:39:59.377051+0000","last_clean":"2024-09-16T00:39:59.377051+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:39:59.377051+0000","last_undegraded":"2024-09-16T00:39:59.377051+0000","last_fullsized":"2024-09-16T00:39:59.377051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377536+0000","last_change":"2024-09-16T00:39:59.376963+0000","last_active":"2024-09-16T00:39:59.377536+0000","last_peered":"2024-09-16T00:39:59.377536+0000","last_clean":"2024-09-16T00:39:59.377536+0000","last_became_active":"2024-09-16T00:39:59.376538+0000","last_became_peered":"2024-09-16T00:39:59.376538+0000","last_unstale":"2024-09-16T00:39:59.377536+0000","last_undegraded":"2024-09-16T00:39:59.377536+0000","last_fullsized":"2024-09-16T00:39:59.377536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:02.077749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371904+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:39:59.371904+0000","last_peered":"2024-09-16T00:39:59.371904+0000","last_clean":"2024-09-16T00:39:59.371904+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:39:59.371904+0000","last_undegraded":"2024-09-16T00:39:59.371904+0000","last_fullsized":"2024-09-16T00:39:59.371904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383392+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:39:59.383392+0000","last_peered":"2024-09-16T00:39:59.383392+0000","last_clean":"2024-09-16T00:39:59.383392+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:39:59.383392+0000","last_undegraded":"2024-09-16T00:39:59.383392+0000","last_fullsized":"2024-09-16T00:39:59.383392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373703+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:39:59.373703+0000","last_peered":"2024-09-16T00:39:59.373703+0000","last_clean":"2024-09-16T00:39:59.373703+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:39:59.373703+0000","last_undegraded":"2024-09-16T00:39:59.373703+0000","last_fullsized":"2024-09-16T00:39:59.373703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371627+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:39:59.371627+0000","last_peered":"2024-09-16T00:39:59.371627+0000","last_clean":"2024-09-16T00:39:59.371627+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:39:59.371627+0000","last_undegraded":"2024-09-16T00:39:59.371627+0000","last_fullsized":"2024-09-16T00:39:59.371627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371307+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:39:59.371307+0000","last_peered":"2024-09-16T00:39:59.371307+0000","last_clean":"2024-09-16T00:39:59.371307+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:39:59.371307+0000","last_undegraded":"2024-09-16T00:39:59.371307+0000","last_fullsized":"2024-09-16T00:39:59.371307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371730+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:39:59.371730+0000","last_peered":"2024-09-16T00:39:59.371730+0000","last_clean":"2024-09-16T00:39:59.371730+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:39:59.371730+0000","last_undegraded":"2024-09-16T00:39:59.371730+0000","last_fullsized":"2024-09-16T00:39:59.371730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.593980+0000","last_change":"2024-09-16T00:39:59.593442+0000","last_active":"2024-09-16T00:39:59.593980+0000","last_peered":"2024-09-16T00:39:59.593980+0000","last_clean":"2024-09-16T00:39:59.593980+0000","last_became_active":"2024-09-16T00:39:59.593039+0000","last_became_peered":"2024-09-16T00:39:59.593039+0000","last_unstale":"2024-09-16T00:39:59.593980+0000","last_undegraded":"2024-09-16T00:39:59.593980+0000","last_fullsized":"2024-09-16T00:39:59.593980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:36.644148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589177+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:39:59.589177+0000","last_peered":"2024-09-16T00:39:59.589177+0000","last_clean":"2024-09-16T00:39:59.589177+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:39:59.589177+0000","last_undegraded":"2024-09-16T00:39:59.589177+0000","last_fullsized":"2024-09-16T00:39:59.589177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371415+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:39:59.371415+0000","last_peered":"2024-09-16T00:39:59.371415+0000","last_clean":"2024-09-16T00:39:59.371415+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:39:59.371415+0000","last_undegraded":"2024-09-16T00:39:59.371415+0000","last_fullsized":"2024-09-16T00:39:59.371415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.372040+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:39:59.372040+0000","last_peered":"2024-09-16T00:39:59.372040+0000","last_clean":"2024-09-16T00:39:59.372040+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:39:59.372040+0000","last_undegraded":"2024-09-16T00:39:59.372040+0000","last_fullsized":"2024-09-16T00:39:59.372040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590273+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:39:59.590273+0000","last_peered":"2024-09-16T00:39:59.590273+0000","last_clean":"2024-09-16T00:39:59.590273+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:39:59.590273+0000","last_undegraded":"2024-09-16T00:39:59.590273+0000","last_fullsized":"2024-09-16T00:39:59.590273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371669+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:39:59.371669+0000","last_peered":"2024-09-16T00:39:59.371669+0000","last_clean":"2024-09-16T00:39:59.371669+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:39:59.371669+0000","last_undegraded":"2024-09-16T00:39:59.371669+0000","last_fullsized":"2024-09-16T00:39:59.371669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.594994+0000","last_change":"2024-09-16T00:39:59.594994+0000","last_active":"2024-09-16T00:39:59.594994+0000","last_peered":"2024-09-16T00:39:59.594994+0000","last_clean":"2024-09-16T00:39:59.594994+0000","last_became_active":"2024-09-16T00:39:59.590640+0000","last_became_peered":"2024-09-16T00:39:59.590640+0000","last_unstale":"2024-09-16T00:39:59.594994+0000","last_undegraded":"2024-09-16T00:39:59.594994+0000","last_fullsized":"2024-09-16T00:39:59.594994+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:21.391494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371160+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:39:59.371160+0000","last_peered":"2024-09-16T00:39:59.371160+0000","last_clean":"2024-09-16T00:39:59.371160+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:39:59.371160+0000","last_undegraded":"2024-09-16T00:39:59.371160+0000","last_fullsized":"2024-09-16T00:39:59.371160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374630+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:39:59.374630+0000","last_peered":"2024-09-16T00:39:59.374630+0000","last_clean":"2024-09-16T00:39:59.374630+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:39:59.374630+0000","last_undegraded":"2024-09-16T00:39:59.374630+0000","last_fullsized":"2024-09-16T00:39:59.374630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.375399+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:39:59.375399+0000","last_peered":"2024-09-16T00:39:59.375399+0000","last_clean":"2024-09-16T00:39:59.375399+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:39:59.375399+0000","last_undegraded":"2024-09-16T00:39:59.375399+0000","last_fullsized":"2024-09-16T00:39:59.375399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371247+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:39:59.371247+0000","last_peered":"2024-09-16T00:39:59.371247+0000","last_clean":"2024-09-16T00:39:59.371247+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:39:59.371247+0000","last_undegraded":"2024-09-16T00:39:59.371247+0000","last_fullsized":"2024-09-16T00:39:59.371247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371341+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:39:59.371341+0000","last_peered":"2024-09-16T00:39:59.371341+0000","last_clean":"2024-09-16T00:39:59.371341+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:39:59.371341+0000","last_undegraded":"2024-09-16T00:39:59.371341+0000","last_fullsized":"2024-09-16T00:39:59.371341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373482+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:39:59.373482+0000","last_peered":"2024-09-16T00:39:59.373482+0000","last_clean":"2024-09-16T00:39:59.373482+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:39:59.373482+0000","last_undegraded":"2024-09-16T00:39:59.373482+0000","last_fullsized":"2024-09-16T00:39:59.373482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377491+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:39:59.377491+0000","last_peered":"2024-09-16T00:39:59.377491+0000","last_clean":"2024-09-16T00:39:59.377491+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:39:59.377491+0000","last_undegraded":"2024-09-16T00:39:59.377491+0000","last_fullsized":"2024-09-16T00:39:59.377491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371119+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:39:59.371119+0000","last_peered":"2024-09-16T00:39:59.371119+0000","last_clean":"2024-09-16T00:39:59.371119+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:39:59.371119+0000","last_undegraded":"2024-09-16T00:39:59.371119+0000","last_fullsized":"2024-09-16T00:39:59.371119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373198+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:39:59.373198+0000","last_peered":"2024-09-16T00:39:59.373198+0000","last_clean":"2024-09-16T00:39:59.373198+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:39:59.373198+0000","last_undegraded":"2024-09-16T00:39:59.373198+0000","last_fullsized":"2024-09-16T00:39:59.373198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.594058+0000","last_change":"2024-09-16T00:39:59.593568+0000","last_active":"2024-09-16T00:39:59.594058+0000","last_peered":"2024-09-16T00:39:59.594058+0000","last_clean":"2024-09-16T00:39:59.594058+0000","last_became_active":"2024-09-16T00:39:59.593209+0000","last_became_peered":"2024-09-16T00:39:59.593209+0000","last_unstale":"2024-09-16T00:39:59.594058+0000","last_undegraded":"2024-09-16T00:39:59.594058+0000","last_fullsized":"2024-09-16T00:39:59.594058+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:56.516773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371238+0000","last_change":"2024-09-16T00:38:52.974330+0000","last_active":"2024-09-16T00:39:59.371238+0000","last_peered":"2024-09-16T00:39:59.371238+0000","last_clean":"2024-09-16T00:39:59.371238+0000","last_became_active":"2024-09-16T00:38:52.974004+0000","last_became_peered":"2024-09-16T00:38:52.974004+0000","last_unstale":"2024-09-16T00:39:59.371238+0000","last_undegraded":"2024-09-16T00:39:59.371238+0000","last_fullsized":"2024-09-16T00:39:59.371238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:31.820720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371467+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:39:59.371467+0000","last_peered":"2024-09-16T00:39:59.371467+0000","last_clean":"2024-09-16T00:39:59.371467+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:39:59.371467+0000","last_undegraded":"2024-09-16T00:39:59.371467+0000","last_fullsized":"2024-09-16T00:39:59.371467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.380258+0000","last_change":"2024-09-16T00:39:59.380258+0000","last_active":"2024-09-16T00:39:59.380258+0000","last_peered":"2024-09-16T00:39:59.380258+0000","last_clean":"2024-09-16T00:39:59.380258+0000","last_became_active":"2024-09-16T00:39:59.379813+0000","last_became_peered":"2024-09-16T00:39:59.379813+0000","last_unstale":"2024-09-16T00:39:59.380258+0000","last_undegraded":"2024-09-16T00:39:59.380258+0000","last_fullsized":"2024-09-16T00:39:59.380258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:03.110305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377545+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:39:59.377545+0000","last_peered":"2024-09-16T00:39:59.377545+0000","last_clean":"2024-09-16T00:39:59.377545+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:39:59.377545+0000","last_undegraded":"2024-09-16T00:39:59.377545+0000","last_fullsized":"2024-09-16T00:39:59.377545+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381185+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:39:59.381185+0000","last_peered":"2024-09-16T00:39:59.381185+0000","last_clean":"2024-09-16T00:39:59.381185+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:39:59.381185+0000","last_undegraded":"2024-09-16T00:39:59.381185+0000","last_fullsized":"2024-09-16T00:39:59.381185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371884+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:39:59.371884+0000","last_peered":"2024-09-16T00:39:59.371884+0000","last_clean":"2024-09-16T00:39:59.371884+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:39:59.371884+0000","last_undegraded":"2024-09-16T00:39:59.371884+0000","last_fullsized":"2024-09-16T00:39:59.371884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374306+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:39:59.374306+0000","last_peered":"2024-09-16T00:39:59.374306+0000","last_clean":"2024-09-16T00:39:59.374306+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:39:59.374306+0000","last_undegraded":"2024-09-16T00:39:59.374306+0000","last_fullsized":"2024-09-16T00:39:59.374306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":175,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:40:25.225224+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:40:25.225224+0000","last_peered":"2024-09-16T00:40:25.225224+0000","last_clean":"2024-09-16T00:40:25.225224+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:40:25.225224+0000","last_undegraded":"2024-09-16T00:40:25.225224+0000","last_fullsized":"2024-09-16T00:40:25.225224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63511,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":111,"num_read_kb":117,"num_write":30,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373436+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:39:59.373436+0000","last_peered":"2024-09-16T00:39:59.373436+0000","last_clean":"2024-09-16T00:39:59.373436+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:39:59.373436+0000","last_undegraded":"2024-09-16T00:39:59.373436+0000","last_fullsized":"2024-09-16T00:39:59.373436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589639+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:39:59.589639+0000","last_peered":"2024-09-16T00:39:59.589639+0000","last_clean":"2024-09-16T00:39:59.589639+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:39:59.589639+0000","last_undegraded":"2024-09-16T00:39:59.589639+0000","last_fullsized":"2024-09-16T00:39:59.589639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.372001+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:39:59.372001+0000","last_peered":"2024-09-16T00:39:59.372001+0000","last_clean":"2024-09-16T00:39:59.372001+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:39:59.372001+0000","last_undegraded":"2024-09-16T00:39:59.372001+0000","last_fullsized":"2024-09-16T00:39:59.372001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373321+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:39:59.373321+0000","last_peered":"2024-09-16T00:39:59.373321+0000","last_clean":"2024-09-16T00:39:59.373321+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:39:59.373321+0000","last_undegraded":"2024-09-16T00:39:59.373321+0000","last_fullsized":"2024-09-16T00:39:59.373321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.592089+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:39:59.592089+0000","last_peered":"2024-09-16T00:39:59.592089+0000","last_clean":"2024-09-16T00:39:59.592089+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:39:59.592089+0000","last_undegraded":"2024-09-16T00:39:59.592089+0000","last_fullsized":"2024-09-16T00:39:59.592089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371515+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:39:59.371515+0000","last_peered":"2024-09-16T00:39:59.371515+0000","last_clean":"2024-09-16T00:39:59.371515+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:39:59.371515+0000","last_undegraded":"2024-09-16T00:39:59.371515+0000","last_fullsized":"2024-09-16T00:39:59.371515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.372652+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:39:59.372652+0000","last_peered":"2024-09-16T00:39:59.372652+0000","last_clean":"2024-09-16T00:39:59.372652+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:39:59.372652+0000","last_undegraded":"2024-09-16T00:39:59.372652+0000","last_fullsized":"2024-09-16T00:39:59.372652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589601+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:39:59.589601+0000","last_peered":"2024-09-16T00:39:59.589601+0000","last_clean":"2024-09-16T00:39:59.589601+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:39:59.589601+0000","last_undegraded":"2024-09-16T00:39:59.589601+0000","last_fullsized":"2024-09-16T00:39:59.589601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590236+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:39:59.590236+0000","last_peered":"2024-09-16T00:39:59.590236+0000","last_clean":"2024-09-16T00:39:59.590236+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:39:59.590236+0000","last_undegraded":"2024-09-16T00:39:59.590236+0000","last_fullsized":"2024-09-16T00:39:59.590236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383032+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:39:59.383032+0000","last_peered":"2024-09-16T00:39:59.383032+0000","last_clean":"2024-09-16T00:39:59.383032+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:39:59.383032+0000","last_undegraded":"2024-09-16T00:39:59.383032+0000","last_fullsized":"2024-09-16T00:39:59.383032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590750+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:39:59.590750+0000","last_peered":"2024-09-16T00:39:59.590750+0000","last_clean":"2024-09-16T00:39:59.590750+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:39:59.590750+0000","last_undegraded":"2024-09-16T00:39:59.590750+0000","last_fullsized":"2024-09-16T00:39:59.590750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373135+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:39:59.373135+0000","last_peered":"2024-09-16T00:39:59.373135+0000","last_clean":"2024-09-16T00:39:59.373135+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:39:59.373135+0000","last_undegraded":"2024-09-16T00:39:59.373135+0000","last_fullsized":"2024-09-16T00:39:59.373135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371503+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:39:59.371503+0000","last_peered":"2024-09-16T00:39:59.371503+0000","last_clean":"2024-09-16T00:39:59.371503+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:39:59.371503+0000","last_undegraded":"2024-09-16T00:39:59.371503+0000","last_fullsized":"2024-09-16T00:39:59.371503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374701+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:39:59.374701+0000","last_peered":"2024-09-16T00:39:59.374701+0000","last_clean":"2024-09-16T00:39:59.374701+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:39:59.374701+0000","last_undegraded":"2024-09-16T00:39:59.374701+0000","last_fullsized":"2024-09-16T00:39:59.374701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381770+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:39:59.381770+0000","last_peered":"2024-09-16T00:39:59.381770+0000","last_clean":"2024-09-16T00:39:59.381770+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:39:59.381770+0000","last_undegraded":"2024-09-16T00:39:59.381770+0000","last_fullsized":"2024-09-16T00:39:59.381770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377157+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:39:59.377157+0000","last_peered":"2024-09-16T00:39:59.377157+0000","last_clean":"2024-09-16T00:39:59.377157+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:39:59.377157+0000","last_undegraded":"2024-09-16T00:39:59.377157+0000","last_fullsized":"2024-09-16T00:39:59.377157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589251+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:39:59.589251+0000","last_peered":"2024-09-16T00:39:59.589251+0000","last_clean":"2024-09-16T00:39:59.589251+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:39:59.589251+0000","last_undegraded":"2024-09-16T00:39:59.589251+0000","last_fullsized":"2024-09-16T00:39:59.589251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371611+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:39:59.371611+0000","last_peered":"2024-09-16T00:39:59.371611+0000","last_clean":"2024-09-16T00:39:59.371611+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:39:59.371611+0000","last_undegraded":"2024-09-16T00:39:59.371611+0000","last_fullsized":"2024-09-16T00:39:59.371611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381316+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:39:59.381316+0000","last_peered":"2024-09-16T00:39:59.381316+0000","last_clean":"2024-09-16T00:39:59.381316+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:39:59.381316+0000","last_undegraded":"2024-09-16T00:39:59.381316+0000","last_fullsized":"2024-09-16T00:39:59.381316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371704+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:39:59.371704+0000","last_peered":"2024-09-16T00:39:59.371704+0000","last_clean":"2024-09-16T00:39:59.371704+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:39:59.371704+0000","last_undegraded":"2024-09-16T00:39:59.371704+0000","last_fullsized":"2024-09-16T00:39:59.371704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373771+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:39:59.373771+0000","last_peered":"2024-09-16T00:39:59.373771+0000","last_clean":"2024-09-16T00:39:59.373771+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:39:59.373771+0000","last_undegraded":"2024-09-16T00:39:59.373771+0000","last_fullsized":"2024-09-16T00:39:59.373771+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381720+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:39:59.381720+0000","last_peered":"2024-09-16T00:39:59.381720+0000","last_clean":"2024-09-16T00:39:59.381720+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:39:59.381720+0000","last_undegraded":"2024-09-16T00:39:59.381720+0000","last_fullsized":"2024-09-16T00:39:59.381720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373893+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:39:59.373893+0000","last_peered":"2024-09-16T00:39:59.373893+0000","last_clean":"2024-09-16T00:39:59.373893+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:39:59.373893+0000","last_undegraded":"2024-09-16T00:39:59.373893+0000","last_fullsized":"2024-09-16T00:39:59.373893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.380959+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:39:59.380959+0000","last_peered":"2024-09-16T00:39:59.380959+0000","last_clean":"2024-09-16T00:39:59.380959+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:39:59.380959+0000","last_undegraded":"2024-09-16T00:39:59.380959+0000","last_fullsized":"2024-09-16T00:39:59.380959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381103+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:39:59.381103+0000","last_peered":"2024-09-16T00:39:59.381103+0000","last_clean":"2024-09-16T00:39:59.381103+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:39:59.381103+0000","last_undegraded":"2024-09-16T00:39:59.381103+0000","last_fullsized":"2024-09-16T00:39:59.381103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590367+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:39:59.590367+0000","last_peered":"2024-09-16T00:39:59.590367+0000","last_clean":"2024-09-16T00:39:59.590367+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:39:59.590367+0000","last_undegraded":"2024-09-16T00:39:59.590367+0000","last_fullsized":"2024-09-16T00:39:59.590367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.378308+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:39:59.378308+0000","last_peered":"2024-09-16T00:39:59.378308+0000","last_clean":"2024-09-16T00:39:59.378308+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:39:59.378308+0000","last_undegraded":"2024-09-16T00:39:59.378308+0000","last_fullsized":"2024-09-16T00:39:59.378308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374557+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:39:59.374557+0000","last_peered":"2024-09-16T00:39:59.374557+0000","last_clean":"2024-09-16T00:39:59.374557+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:39:59.374557+0000","last_undegraded":"2024-09-16T00:39:59.374557+0000","last_fullsized":"2024-09-16T00:39:59.374557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381810+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:39:59.381810+0000","last_peered":"2024-09-16T00:39:59.381810+0000","last_clean":"2024-09-16T00:39:59.381810+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:39:59.381810+0000","last_undegraded":"2024-09-16T00:39:59.381810+0000","last_fullsized":"2024-09-16T00:39:59.381810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383559+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:39:59.383559+0000","last_peered":"2024-09-16T00:39:59.383559+0000","last_clean":"2024-09-16T00:39:59.383559+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:39:59.383559+0000","last_undegraded":"2024-09-16T00:39:59.383559+0000","last_fullsized":"2024-09-16T00:39:59.383559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371269+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:39:59.371269+0000","last_peered":"2024-09-16T00:39:59.371269+0000","last_clean":"2024-09-16T00:39:59.371269+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:39:59.371269+0000","last_undegraded":"2024-09-16T00:39:59.371269+0000","last_fullsized":"2024-09-16T00:39:59.371269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.382780+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:39:59.382780+0000","last_peered":"2024-09-16T00:39:59.382780+0000","last_clean":"2024-09-16T00:39:59.382780+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:39:59.382780+0000","last_undegraded":"2024-09-16T00:39:59.382780+0000","last_fullsized":"2024-09-16T00:39:59.382780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589955+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:39:59.589955+0000","last_peered":"2024-09-16T00:39:59.589955+0000","last_clean":"2024-09-16T00:39:59.589955+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:39:59.589955+0000","last_undegraded":"2024-09-16T00:39:59.589955+0000","last_fullsized":"2024-09-16T00:39:59.589955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.378063+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:39:59.378063+0000","last_peered":"2024-09-16T00:39:59.378063+0000","last_clean":"2024-09-16T00:39:59.378063+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:39:59.378063+0000","last_undegraded":"2024-09-16T00:39:59.378063+0000","last_fullsized":"2024-09-16T00:39:59.378063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373932+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:39:59.373932+0000","last_peered":"2024-09-16T00:39:59.373932+0000","last_clean":"2024-09-16T00:39:59.373932+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:39:59.373932+0000","last_undegraded":"2024-09-16T00:39:59.373932+0000","last_fullsized":"2024-09-16T00:39:59.373932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381657+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:39:59.381657+0000","last_peered":"2024-09-16T00:39:59.381657+0000","last_clean":"2024-09-16T00:39:59.381657+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:39:59.381657+0000","last_undegraded":"2024-09-16T00:39:59.381657+0000","last_fullsized":"2024-09-16T00:39:59.381657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374260+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:39:59.374260+0000","last_peered":"2024-09-16T00:39:59.374260+0000","last_clean":"2024-09-16T00:39:59.374260+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:39:59.374260+0000","last_undegraded":"2024-09-16T00:39:59.374260+0000","last_fullsized":"2024-09-16T00:39:59.374260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374983+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:39:59.374983+0000","last_peered":"2024-09-16T00:39:59.374983+0000","last_clean":"2024-09-16T00:39:59.374983+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:39:59.374983+0000","last_undegraded":"2024-09-16T00:39:59.374983+0000","last_fullsized":"2024-09-16T00:39:59.374983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589031+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:39:59.589031+0000","last_peered":"2024-09-16T00:39:59.589031+0000","last_clean":"2024-09-16T00:39:59.589031+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:39:59.589031+0000","last_undegraded":"2024-09-16T00:39:59.589031+0000","last_fullsized":"2024-09-16T00:39:59.589031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589750+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:39:59.589750+0000","last_peered":"2024-09-16T00:39:59.589750+0000","last_clean":"2024-09-16T00:39:59.589750+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:39:59.589750+0000","last_undegraded":"2024-09-16T00:39:59.589750+0000","last_fullsized":"2024-09-16T00:39:59.589750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589987+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:39:59.589987+0000","last_peered":"2024-09-16T00:39:59.589987+0000","last_clean":"2024-09-16T00:39:59.589987+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:39:59.589987+0000","last_undegraded":"2024-09-16T00:39:59.589987+0000","last_fullsized":"2024-09-16T00:39:59.589987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377978+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:39:59.377978+0000","last_peered":"2024-09-16T00:39:59.377978+0000","last_clean":"2024-09-16T00:39:59.377978+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:39:59.377978+0000","last_undegraded":"2024-09-16T00:39:59.377978+0000","last_fullsized":"2024-09-16T00:39:59.377978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374112+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:39:59.374112+0000","last_peered":"2024-09-16T00:39:59.374112+0000","last_clean":"2024-09-16T00:39:59.374112+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:39:59.374112+0000","last_undegraded":"2024-09-16T00:39:59.374112+0000","last_fullsized":"2024-09-16T00:39:59.374112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371544+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:39:59.371544+0000","last_peered":"2024-09-16T00:39:59.371544+0000","last_clean":"2024-09-16T00:39:59.371544+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:39:59.371544+0000","last_undegraded":"2024-09-16T00:39:59.371544+0000","last_fullsized":"2024-09-16T00:39:59.371544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590122+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:39:59.590122+0000","last_peered":"2024-09-16T00:39:59.590122+0000","last_clean":"2024-09-16T00:39:59.590122+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:39:59.590122+0000","last_undegraded":"2024-09-16T00:39:59.590122+0000","last_fullsized":"2024-09-16T00:39:59.590122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589500+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:39:59.589500+0000","last_peered":"2024-09-16T00:39:59.589500+0000","last_clean":"2024-09-16T00:39:59.589500+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:39:59.589500+0000","last_undegraded":"2024-09-16T00:39:59.589500+0000","last_fullsized":"2024-09-16T00:39:59.589500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381674+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:39:59.381674+0000","last_peered":"2024-09-16T00:39:59.381674+0000","last_clean":"2024-09-16T00:39:59.381674+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:39:59.381674+0000","last_undegraded":"2024-09-16T00:39:59.381674+0000","last_fullsized":"2024-09-16T00:39:59.381674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371065+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:39:59.371065+0000","last_peered":"2024-09-16T00:39:59.371065+0000","last_clean":"2024-09-16T00:39:59.371065+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:39:59.371065+0000","last_undegraded":"2024-09-16T00:39:59.371065+0000","last_fullsized":"2024-09-16T00:39:59.371065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589133+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:39:59.589133+0000","last_peered":"2024-09-16T00:39:59.589133+0000","last_clean":"2024-09-16T00:39:59.589133+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:39:59.589133+0000","last_undegraded":"2024-09-16T00:39:59.589133+0000","last_fullsized":"2024-09-16T00:39:59.589133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373580+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:39:59.373580+0000","last_peered":"2024-09-16T00:39:59.373580+0000","last_clean":"2024-09-16T00:39:59.373580+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:39:59.373580+0000","last_undegraded":"2024-09-16T00:39:59.373580+0000","last_fullsized":"2024-09-16T00:39:59.373580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374059+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:39:59.374059+0000","last_peered":"2024-09-16T00:39:59.374059+0000","last_clean":"2024-09-16T00:39:59.374059+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:39:59.374059+0000","last_undegraded":"2024-09-16T00:39:59.374059+0000","last_fullsized":"2024-09-16T00:39:59.374059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590548+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:39:59.590548+0000","last_peered":"2024-09-16T00:39:59.590548+0000","last_clean":"2024-09-16T00:39:59.590548+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:39:59.590548+0000","last_undegraded":"2024-09-16T00:39:59.590548+0000","last_fullsized":"2024-09-16T00:39:59.590548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590849+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:39:59.590849+0000","last_peered":"2024-09-16T00:39:59.590849+0000","last_clean":"2024-09-16T00:39:59.590849+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:39:59.590849+0000","last_undegraded":"2024-09-16T00:39:59.590849+0000","last_fullsized":"2024-09-16T00:39:59.590849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374940+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:39:59.374940+0000","last_peered":"2024-09-16T00:39:59.374940+0000","last_clean":"2024-09-16T00:39:59.374940+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:39:59.374940+0000","last_undegraded":"2024-09-16T00:39:59.374940+0000","last_fullsized":"2024-09-16T00:39:59.374940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64309,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":117,"num_read_kb":123,"num_write":85,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":192927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547233,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1003520,"data_stored":499483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36992,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708160,"statfs":{"total":95995035648,"available":95957155840,"internally_reserved":0,"allocated":1683456,"data_stored":1153556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939690,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":500065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":500087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1605632,"data_stored":1094529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561082,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":495387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1609728,"data_stored":1090033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":499505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":495491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444111,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1077248,"data_stored":558932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1073152,"data_stored":558898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1003520,"data_stored":495477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:40:30.105 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:40:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:30.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:30 smithi081 ceph-mon[35539]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:40:30.769 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:30 smithi031 ceph-mon[30956]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:40:30.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:30 smithi102 ceph-mon[35364]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:40:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 -- 172.21.15.31:0/1110058148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd141002f0 msgr2=0x7fdd141006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 --2- 172.21.15.31:0/1110058148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd141002f0 0x7fdd141006d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009930 tx=0x7fdd0402a420 comp rx=0 tx=0).stop 2024-09-16T00:40:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 -- 172.21.15.31:0/1110058148 shutdown_connections 2024-09-16T00:40:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 --2- 172.21.15.31:0/1110058148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd14106f50 0x7fdd1410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 --2- 172.21.15.31:0/1110058148 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd14100ca0 0x7fdd14106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 --2- 172.21.15.31:0/1110058148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd141002f0 0x7fdd141006d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 -- 172.21.15.31:0/1110058148 >> 172.21.15.31:0/1110058148 conn(0x7fdd14077210 msgr2=0x7fdd140fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 -- 172.21.15.31:0/1110058148 shutdown_connections 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 -- 172.21.15.31:0/1110058148 wait complete. 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 Processor -- start 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.852+0000 7fdd1a682640 1 -- start start 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd1a682640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd141002f0 0x7fdd14101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd1a682640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd14100ca0 0x7fdd141017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd1a682640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14106f50 0x7fdd14101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd1a682640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd14112000 con 0x7fdd14106f50 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd1a682640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdd14111e80 con 0x7fdd141002f0 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd1a682640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdd14112180 con 0x7fdd14100ca0 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd18c1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14106f50 0x7fdd14101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd18c1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14106f50 0x7fdd14101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49820/0 (socket says 172.21.15.31:49820) 2024-09-16T00:40:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd18c1f640 1 -- 172.21.15.31:0/136475251 learned_addr learned my addr 172.21.15.31:0/136475251 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd13fff640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd141002f0 0x7fdd14101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd137fe640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd14100ca0 0x7fdd141017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd18c1f640 1 -- 172.21.15.31:0/136475251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd14100ca0 msgr2=0x7fdd141017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd18c1f640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd14100ca0 0x7fdd141017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd18c1f640 1 -- 172.21.15.31:0/136475251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd141002f0 msgr2=0x7fdd14101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd18c1f640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd141002f0 0x7fdd14101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd18c1f640 1 -- 172.21.15.31:0/136475251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd141a6a60 con 0x7fdd14106f50 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd137fe640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd14100ca0 0x7fdd141017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.853+0000 7fdd18c1f640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14106f50 0x7fdd14101cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00b970 tx=0x7fdd0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.854+0000 7fdd117fa640 1 -- 172.21.15.31:0/136475251 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c019070 con 0x7fdd14106f50 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.854+0000 7fdd117fa640 1 -- 172.21.15.31:0/136475251 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0c00cb40 con 0x7fdd14106f50 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.854+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd141a6d50 con 0x7fdd14106f50 2024-09-16T00:40:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.854+0000 7fdd117fa640 1 -- 172.21.15.31:0/136475251 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c007650 con 0x7fdd14106f50 2024-09-16T00:40:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.854+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd141a7240 con 0x7fdd14106f50 2024-09-16T00:40:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.855+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd141006d0 con 0x7fdd14106f50 2024-09-16T00:40:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.856+0000 7fdd117fa640 1 -- 172.21.15.31:0/136475251 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdd0c01c070 con 0x7fdd14106f50 2024-09-16T00:40:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.856+0000 7fdd117fa640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcfc0a32d0 0x7fdcfc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.857+0000 7fdd13fff640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcfc0a32d0 0x7fdcfc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.857+0000 7fdd117fa640 1 -- 172.21.15.31:0/136475251 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fdd0c0cf280 con 0x7fdd14106f50 2024-09-16T00:40:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.857+0000 7fdd13fff640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcfc0a32d0 0x7fdcfc0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009900 tx=0x7fdd040058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:30.861+0000 7fdd117fa640 1 -- 172.21.15.31:0/136475251 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd0c09b5d0 con 0x7fdd14106f50 2024-09-16T00:40:31.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.036+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd1410a1b0 con 0x7fdcfc0a32d0 2024-09-16T00:40:31.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.044+0000 7fdd117fa640 1 -- 172.21.15.31:0/136475251 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340285 (secure 0 0 0) 0x7fdd04031860 con 0x7fdcfc0a32d0 2024-09-16T00:40:31.048 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:31.050 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:40:31.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcfc0a32d0 msgr2=0x7fdcfc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:31.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcfc0a32d0 0x7fdcfc0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009900 tx=0x7fdd040058d0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14106f50 msgr2=0x7fdd14101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:31.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14106f50 0x7fdd14101cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00b970 tx=0x7fdd0c00be30 comp rx=0 tx=0).stop 2024-09-16T00:40:31.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 shutdown_connections 2024-09-16T00:40:31.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd14106f50 0x7fdd14101cf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcfc0a32d0 0x7fdcfc0a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd14100ca0 0x7fdd141017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 --2- 172.21.15.31:0/136475251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd141002f0 0x7fdd14101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:31.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 >> 172.21.15.31:0/136475251 conn(0x7fdd14077210 msgr2=0x7fdd141052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:31.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 shutdown_connections 2024-09-16T00:40:31.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:31.048+0000 7fdd1a682640 1 -- 172.21.15.31:0/136475251 wait complete. 2024-09-16T00:40:31.674 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:31 smithi031 ceph-mon[30956]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:31.704 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-16T00:40:29.675964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654911,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":243,"num_read_kb":232,"num_write":268,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400524,"kb_used_data":13760,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541300,"statfs":{"total":1151940427776,"available":1151530291200,"internally_reserved":0,"allocated":14090240,"data_stored":7941443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002974"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373754+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:39:59.373754+0000","last_peered":"2024-09-16T00:39:59.373754+0000","last_clean":"2024-09-16T00:39:59.373754+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:39:59.373754+0000","last_undegraded":"2024-09-16T00:39:59.373754+0000","last_fullsized":"2024-09-16T00:39:59.373754+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383882+0000","last_change":"2024-09-16T00:38:52.968681+0000","last_active":"2024-09-16T00:39:59.383882+0000","last_peered":"2024-09-16T00:39:59.383882+0000","last_clean":"2024-09-16T00:39:59.383882+0000","last_became_active":"2024-09-16T00:38:52.968359+0000","last_became_peered":"2024-09-16T00:38:52.968359+0000","last_unstale":"2024-09-16T00:39:59.383882+0000","last_undegraded":"2024-09-16T00:39:59.383882+0000","last_fullsized":"2024-09-16T00:39:59.383882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:17.870176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.380513+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:39:59.380513+0000","last_peered":"2024-09-16T00:39:59.380513+0000","last_clean":"2024-09-16T00:39:59.380513+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:39:59.380513+0000","last_undegraded":"2024-09-16T00:39:59.380513+0000","last_fullsized":"2024-09-16T00:39:59.380513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.384005+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:39:59.384005+0000","last_peered":"2024-09-16T00:39:59.384005+0000","last_clean":"2024-09-16T00:39:59.384005+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:39:59.384005+0000","last_undegraded":"2024-09-16T00:39:59.384005+0000","last_fullsized":"2024-09-16T00:39:59.384005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.591997+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:39:59.591997+0000","last_peered":"2024-09-16T00:39:59.591997+0000","last_clean":"2024-09-16T00:39:59.591997+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:39:59.591997+0000","last_undegraded":"2024-09-16T00:39:59.591997+0000","last_fullsized":"2024-09-16T00:39:59.591997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.372931+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:39:59.372931+0000","last_peered":"2024-09-16T00:39:59.372931+0000","last_clean":"2024-09-16T00:39:59.372931+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:39:59.372931+0000","last_undegraded":"2024-09-16T00:39:59.372931+0000","last_fullsized":"2024-09-16T00:39:59.372931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374941+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:39:59.374941+0000","last_peered":"2024-09-16T00:39:59.374941+0000","last_clean":"2024-09-16T00:39:59.374941+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:39:59.374941+0000","last_undegraded":"2024-09-16T00:39:59.374941+0000","last_fullsized":"2024-09-16T00:39:59.374941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589523+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:39:59.589523+0000","last_peered":"2024-09-16T00:39:59.589523+0000","last_clean":"2024-09-16T00:39:59.589523+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:39:59.589523+0000","last_undegraded":"2024-09-16T00:39:59.589523+0000","last_fullsized":"2024-09-16T00:39:59.589523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373244+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:39:59.373244+0000","last_peered":"2024-09-16T00:39:59.373244+0000","last_clean":"2024-09-16T00:39:59.373244+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:39:59.373244+0000","last_undegraded":"2024-09-16T00:39:59.373244+0000","last_fullsized":"2024-09-16T00:39:59.373244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374246+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:39:59.374246+0000","last_peered":"2024-09-16T00:39:59.374246+0000","last_clean":"2024-09-16T00:39:59.374246+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:39:59.374246+0000","last_undegraded":"2024-09-16T00:39:59.374246+0000","last_fullsized":"2024-09-16T00:39:59.374246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371819+0000","last_change":"2024-09-16T00:38:52.975634+0000","last_active":"2024-09-16T00:39:59.371819+0000","last_peered":"2024-09-16T00:39:59.371819+0000","last_clean":"2024-09-16T00:39:59.371819+0000","last_became_active":"2024-09-16T00:38:52.975314+0000","last_became_peered":"2024-09-16T00:38:52.975314+0000","last_unstale":"2024-09-16T00:39:59.371819+0000","last_undegraded":"2024-09-16T00:39:59.371819+0000","last_fullsized":"2024-09-16T00:39:59.371819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:28.716600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373159+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:39:59.373159+0000","last_peered":"2024-09-16T00:39:59.373159+0000","last_clean":"2024-09-16T00:39:59.373159+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:39:59.373159+0000","last_undegraded":"2024-09-16T00:39:59.373159+0000","last_fullsized":"2024-09-16T00:39:59.373159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371178+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:39:59.371178+0000","last_peered":"2024-09-16T00:39:59.371178+0000","last_clean":"2024-09-16T00:39:59.371178+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:39:59.371178+0000","last_undegraded":"2024-09-16T00:39:59.371178+0000","last_fullsized":"2024-09-16T00:39:59.371178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373680+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:39:59.373680+0000","last_peered":"2024-09-16T00:39:59.373680+0000","last_clean":"2024-09-16T00:39:59.373680+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:39:59.373680+0000","last_undegraded":"2024-09-16T00:39:59.373680+0000","last_fullsized":"2024-09-16T00:39:59.373680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373320+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:39:59.373320+0000","last_peered":"2024-09-16T00:39:59.373320+0000","last_clean":"2024-09-16T00:39:59.373320+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:39:59.373320+0000","last_undegraded":"2024-09-16T00:39:59.373320+0000","last_fullsized":"2024-09-16T00:39:59.373320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381978+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:39:59.381978+0000","last_peered":"2024-09-16T00:39:59.381978+0000","last_clean":"2024-09-16T00:39:59.381978+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:39:59.381978+0000","last_undegraded":"2024-09-16T00:39:59.381978+0000","last_fullsized":"2024-09-16T00:39:59.381978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.591565+0000","last_change":"2024-09-16T00:39:59.591063+0000","last_active":"2024-09-16T00:39:59.591565+0000","last_peered":"2024-09-16T00:39:59.591565+0000","last_clean":"2024-09-16T00:39:59.591565+0000","last_became_active":"2024-09-16T00:39:59.590574+0000","last_became_peered":"2024-09-16T00:39:59.590574+0000","last_unstale":"2024-09-16T00:39:59.591565+0000","last_undegraded":"2024-09-16T00:39:59.591565+0000","last_fullsized":"2024-09-16T00:39:59.591565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:57.020592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.595772+0000","last_change":"2024-09-16T00:39:59.595772+0000","last_active":"2024-09-16T00:39:59.595772+0000","last_peered":"2024-09-16T00:39:59.595772+0000","last_clean":"2024-09-16T00:39:59.595772+0000","last_became_active":"2024-09-16T00:39:59.590772+0000","last_became_peered":"2024-09-16T00:39:59.590772+0000","last_unstale":"2024-09-16T00:39:59.595772+0000","last_undegraded":"2024-09-16T00:39:59.595772+0000","last_fullsized":"2024-09-16T00:39:59.595772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:53.236019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373961+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:39:59.373961+0000","last_peered":"2024-09-16T00:39:59.373961+0000","last_clean":"2024-09-16T00:39:59.373961+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:39:59.373961+0000","last_undegraded":"2024-09-16T00:39:59.373961+0000","last_fullsized":"2024-09-16T00:39:59.373961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.382933+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:39:59.382933+0000","last_peered":"2024-09-16T00:39:59.382933+0000","last_clean":"2024-09-16T00:39:59.382933+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:39:59.382933+0000","last_undegraded":"2024-09-16T00:39:59.382933+0000","last_fullsized":"2024-09-16T00:39:59.382933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.386827+0000","last_change":"2024-09-16T00:39:59.386827+0000","last_active":"2024-09-16T00:39:59.386827+0000","last_peered":"2024-09-16T00:39:59.386827+0000","last_clean":"2024-09-16T00:39:59.386827+0000","last_became_active":"2024-09-16T00:39:59.386389+0000","last_became_peered":"2024-09-16T00:39:59.386389+0000","last_unstale":"2024-09-16T00:39:59.386827+0000","last_undegraded":"2024-09-16T00:39:59.386827+0000","last_fullsized":"2024-09-16T00:39:59.386827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:12:21.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373450+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:39:59.373450+0000","last_peered":"2024-09-16T00:39:59.373450+0000","last_clean":"2024-09-16T00:39:59.373450+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:39:59.373450+0000","last_undegraded":"2024-09-16T00:39:59.373450+0000","last_fullsized":"2024-09-16T00:39:59.373450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589918+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:39:59.589918+0000","last_peered":"2024-09-16T00:39:59.589918+0000","last_clean":"2024-09-16T00:39:59.589918+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:39:59.589918+0000","last_undegraded":"2024-09-16T00:39:59.589918+0000","last_fullsized":"2024-09-16T00:39:59.589918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373345+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:39:59.373345+0000","last_peered":"2024-09-16T00:39:59.373345+0000","last_clean":"2024-09-16T00:39:59.373345+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:39:59.373345+0000","last_undegraded":"2024-09-16T00:39:59.373345+0000","last_fullsized":"2024-09-16T00:39:59.373345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590946+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:39:59.590946+0000","last_peered":"2024-09-16T00:39:59.590946+0000","last_clean":"2024-09-16T00:39:59.590946+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:39:59.590946+0000","last_undegraded":"2024-09-16T00:39:59.590946+0000","last_fullsized":"2024-09-16T00:39:59.590946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373627+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:39:59.373627+0000","last_peered":"2024-09-16T00:39:59.373627+0000","last_clean":"2024-09-16T00:39:59.373627+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:39:59.373627+0000","last_undegraded":"2024-09-16T00:39:59.373627+0000","last_fullsized":"2024-09-16T00:39:59.373627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.380670+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:39:59.380670+0000","last_peered":"2024-09-16T00:39:59.380670+0000","last_clean":"2024-09-16T00:39:59.380670+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:39:59.380670+0000","last_undegraded":"2024-09-16T00:39:59.380670+0000","last_fullsized":"2024-09-16T00:39:59.380670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383652+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:39:59.383652+0000","last_peered":"2024-09-16T00:39:59.383652+0000","last_clean":"2024-09-16T00:39:59.383652+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:39:59.383652+0000","last_undegraded":"2024-09-16T00:39:59.383652+0000","last_fullsized":"2024-09-16T00:39:59.383652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374360+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:39:59.374360+0000","last_peered":"2024-09-16T00:39:59.374360+0000","last_clean":"2024-09-16T00:39:59.374360+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:39:59.374360+0000","last_undegraded":"2024-09-16T00:39:59.374360+0000","last_fullsized":"2024-09-16T00:39:59.374360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381531+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:39:59.381531+0000","last_peered":"2024-09-16T00:39:59.381531+0000","last_clean":"2024-09-16T00:39:59.381531+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:39:59.381531+0000","last_undegraded":"2024-09-16T00:39:59.381531+0000","last_fullsized":"2024-09-16T00:39:59.381531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.378161+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:39:59.378161+0000","last_peered":"2024-09-16T00:39:59.378161+0000","last_clean":"2024-09-16T00:39:59.378161+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:39:59.378161+0000","last_undegraded":"2024-09-16T00:39:59.378161+0000","last_fullsized":"2024-09-16T00:39:59.378161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.375058+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:39:59.375058+0000","last_peered":"2024-09-16T00:39:59.375058+0000","last_clean":"2024-09-16T00:39:59.375058+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:39:59.375058+0000","last_undegraded":"2024-09-16T00:39:59.375058+0000","last_fullsized":"2024-09-16T00:39:59.375058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374082+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:39:59.374082+0000","last_peered":"2024-09-16T00:39:59.374082+0000","last_clean":"2024-09-16T00:39:59.374082+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:39:59.374082+0000","last_undegraded":"2024-09-16T00:39:59.374082+0000","last_fullsized":"2024-09-16T00:39:59.374082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589288+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:39:59.589288+0000","last_peered":"2024-09-16T00:39:59.589288+0000","last_clean":"2024-09-16T00:39:59.589288+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:39:59.589288+0000","last_undegraded":"2024-09-16T00:39:59.589288+0000","last_fullsized":"2024-09-16T00:39:59.589288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374366+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:39:59.374366+0000","last_peered":"2024-09-16T00:39:59.374366+0000","last_clean":"2024-09-16T00:39:59.374366+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:39:59.374366+0000","last_undegraded":"2024-09-16T00:39:59.374366+0000","last_fullsized":"2024-09-16T00:39:59.374366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373604+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:39:59.373604+0000","last_peered":"2024-09-16T00:39:59.373604+0000","last_clean":"2024-09-16T00:39:59.373604+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:39:59.373604+0000","last_undegraded":"2024-09-16T00:39:59.373604+0000","last_fullsized":"2024-09-16T00:39:59.373604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589506+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:39:59.589506+0000","last_peered":"2024-09-16T00:39:59.589506+0000","last_clean":"2024-09-16T00:39:59.589506+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:39:59.589506+0000","last_undegraded":"2024-09-16T00:39:59.589506+0000","last_fullsized":"2024-09-16T00:39:59.589506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371437+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:39:59.371437+0000","last_peered":"2024-09-16T00:39:59.371437+0000","last_clean":"2024-09-16T00:39:59.371437+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:39:59.371437+0000","last_undegraded":"2024-09-16T00:39:59.371437+0000","last_fullsized":"2024-09-16T00:39:59.371437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377209+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:39:59.377209+0000","last_peered":"2024-09-16T00:39:59.377209+0000","last_clean":"2024-09-16T00:39:59.377209+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:39:59.377209+0000","last_undegraded":"2024-09-16T00:39:59.377209+0000","last_fullsized":"2024-09-16T00:39:59.377209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371391+0000","last_change":"2024-09-16T00:38:52.972145+0000","last_active":"2024-09-16T00:39:59.371391+0000","last_peered":"2024-09-16T00:39:59.371391+0000","last_clean":"2024-09-16T00:39:59.371391+0000","last_became_active":"2024-09-16T00:38:52.971847+0000","last_became_peered":"2024-09-16T00:38:52.971847+0000","last_unstale":"2024-09-16T00:39:59.371391+0000","last_undegraded":"2024-09-16T00:39:59.371391+0000","last_fullsized":"2024-09-16T00:39:59.371391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:32.819389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377252+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:39:59.377252+0000","last_peered":"2024-09-16T00:39:59.377252+0000","last_clean":"2024-09-16T00:39:59.377252+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:39:59.377252+0000","last_undegraded":"2024-09-16T00:39:59.377252+0000","last_fullsized":"2024-09-16T00:39:59.377252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.378369+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:39:59.378369+0000","last_peered":"2024-09-16T00:39:59.378369+0000","last_clean":"2024-09-16T00:39:59.378369+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:39:59.378369+0000","last_undegraded":"2024-09-16T00:39:59.378369+0000","last_fullsized":"2024-09-16T00:39:59.378369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377051+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:39:59.377051+0000","last_peered":"2024-09-16T00:39:59.377051+0000","last_clean":"2024-09-16T00:39:59.377051+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:39:59.377051+0000","last_undegraded":"2024-09-16T00:39:59.377051+0000","last_fullsized":"2024-09-16T00:39:59.377051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377536+0000","last_change":"2024-09-16T00:39:59.376963+0000","last_active":"2024-09-16T00:39:59.377536+0000","last_peered":"2024-09-16T00:39:59.377536+0000","last_clean":"2024-09-16T00:39:59.377536+0000","last_became_active":"2024-09-16T00:39:59.376538+0000","last_became_peered":"2024-09-16T00:39:59.376538+0000","last_unstale":"2024-09-16T00:39:59.377536+0000","last_undegraded":"2024-09-16T00:39:59.377536+0000","last_fullsized":"2024-09-16T00:39:59.377536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:02.077749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371904+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:39:59.371904+0000","last_peered":"2024-09-16T00:39:59.371904+0000","last_clean":"2024-09-16T00:39:59.371904+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:39:59.371904+0000","last_undegraded":"2024-09-16T00:39:59.371904+0000","last_fullsized":"2024-09-16T00:39:59.371904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383392+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:39:59.383392+0000","last_peered":"2024-09-16T00:39:59.383392+0000","last_clean":"2024-09-16T00:39:59.383392+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:39:59.383392+0000","last_undegraded":"2024-09-16T00:39:59.383392+0000","last_fullsized":"2024-09-16T00:39:59.383392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373703+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:39:59.373703+0000","last_peered":"2024-09-16T00:39:59.373703+0000","last_clean":"2024-09-16T00:39:59.373703+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:39:59.373703+0000","last_undegraded":"2024-09-16T00:39:59.373703+0000","last_fullsized":"2024-09-16T00:39:59.373703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371627+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:39:59.371627+0000","last_peered":"2024-09-16T00:39:59.371627+0000","last_clean":"2024-09-16T00:39:59.371627+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:39:59.371627+0000","last_undegraded":"2024-09-16T00:39:59.371627+0000","last_fullsized":"2024-09-16T00:39:59.371627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371307+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:39:59.371307+0000","last_peered":"2024-09-16T00:39:59.371307+0000","last_clean":"2024-09-16T00:39:59.371307+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:39:59.371307+0000","last_undegraded":"2024-09-16T00:39:59.371307+0000","last_fullsized":"2024-09-16T00:39:59.371307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371730+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:39:59.371730+0000","last_peered":"2024-09-16T00:39:59.371730+0000","last_clean":"2024-09-16T00:39:59.371730+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:39:59.371730+0000","last_undegraded":"2024-09-16T00:39:59.371730+0000","last_fullsized":"2024-09-16T00:39:59.371730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.593980+0000","last_change":"2024-09-16T00:39:59.593442+0000","last_active":"2024-09-16T00:39:59.593980+0000","last_peered":"2024-09-16T00:39:59.593980+0000","last_clean":"2024-09-16T00:39:59.593980+0000","last_became_active":"2024-09-16T00:39:59.593039+0000","last_became_peered":"2024-09-16T00:39:59.593039+0000","last_unstale":"2024-09-16T00:39:59.593980+0000","last_undegraded":"2024-09-16T00:39:59.593980+0000","last_fullsized":"2024-09-16T00:39:59.593980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:36.644148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589177+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:39:59.589177+0000","last_peered":"2024-09-16T00:39:59.589177+0000","last_clean":"2024-09-16T00:39:59.589177+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:39:59.589177+0000","last_undegraded":"2024-09-16T00:39:59.589177+0000","last_fullsized":"2024-09-16T00:39:59.589177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371415+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:39:59.371415+0000","last_peered":"2024-09-16T00:39:59.371415+0000","last_clean":"2024-09-16T00:39:59.371415+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:39:59.371415+0000","last_undegraded":"2024-09-16T00:39:59.371415+0000","last_fullsized":"2024-09-16T00:39:59.371415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.372040+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:39:59.372040+0000","last_peered":"2024-09-16T00:39:59.372040+0000","last_clean":"2024-09-16T00:39:59.372040+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:39:59.372040+0000","last_undegraded":"2024-09-16T00:39:59.372040+0000","last_fullsized":"2024-09-16T00:39:59.372040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590273+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:39:59.590273+0000","last_peered":"2024-09-16T00:39:59.590273+0000","last_clean":"2024-09-16T00:39:59.590273+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:39:59.590273+0000","last_undegraded":"2024-09-16T00:39:59.590273+0000","last_fullsized":"2024-09-16T00:39:59.590273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371669+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:39:59.371669+0000","last_peered":"2024-09-16T00:39:59.371669+0000","last_clean":"2024-09-16T00:39:59.371669+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:39:59.371669+0000","last_undegraded":"2024-09-16T00:39:59.371669+0000","last_fullsized":"2024-09-16T00:39:59.371669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.594994+0000","last_change":"2024-09-16T00:39:59.594994+0000","last_active":"2024-09-16T00:39:59.594994+0000","last_peered":"2024-09-16T00:39:59.594994+0000","last_clean":"2024-09-16T00:39:59.594994+0000","last_became_active":"2024-09-16T00:39:59.590640+0000","last_became_peered":"2024-09-16T00:39:59.590640+0000","last_unstale":"2024-09-16T00:39:59.594994+0000","last_undegraded":"2024-09-16T00:39:59.594994+0000","last_fullsized":"2024-09-16T00:39:59.594994+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:21.391494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371160+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:39:59.371160+0000","last_peered":"2024-09-16T00:39:59.371160+0000","last_clean":"2024-09-16T00:39:59.371160+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:39:59.371160+0000","last_undegraded":"2024-09-16T00:39:59.371160+0000","last_fullsized":"2024-09-16T00:39:59.371160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374630+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:39:59.374630+0000","last_peered":"2024-09-16T00:39:59.374630+0000","last_clean":"2024-09-16T00:39:59.374630+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:39:59.374630+0000","last_undegraded":"2024-09-16T00:39:59.374630+0000","last_fullsized":"2024-09-16T00:39:59.374630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.375399+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:39:59.375399+0000","last_peered":"2024-09-16T00:39:59.375399+0000","last_clean":"2024-09-16T00:39:59.375399+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:39:59.375399+0000","last_undegraded":"2024-09-16T00:39:59.375399+0000","last_fullsized":"2024-09-16T00:39:59.375399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371247+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:39:59.371247+0000","last_peered":"2024-09-16T00:39:59.371247+0000","last_clean":"2024-09-16T00:39:59.371247+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:39:59.371247+0000","last_undegraded":"2024-09-16T00:39:59.371247+0000","last_fullsized":"2024-09-16T00:39:59.371247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371341+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:39:59.371341+0000","last_peered":"2024-09-16T00:39:59.371341+0000","last_clean":"2024-09-16T00:39:59.371341+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:39:59.371341+0000","last_undegraded":"2024-09-16T00:39:59.371341+0000","last_fullsized":"2024-09-16T00:39:59.371341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373482+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:39:59.373482+0000","last_peered":"2024-09-16T00:39:59.373482+0000","last_clean":"2024-09-16T00:39:59.373482+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:39:59.373482+0000","last_undegraded":"2024-09-16T00:39:59.373482+0000","last_fullsized":"2024-09-16T00:39:59.373482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377491+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:39:59.377491+0000","last_peered":"2024-09-16T00:39:59.377491+0000","last_clean":"2024-09-16T00:39:59.377491+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:39:59.377491+0000","last_undegraded":"2024-09-16T00:39:59.377491+0000","last_fullsized":"2024-09-16T00:39:59.377491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371119+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:39:59.371119+0000","last_peered":"2024-09-16T00:39:59.371119+0000","last_clean":"2024-09-16T00:39:59.371119+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:39:59.371119+0000","last_undegraded":"2024-09-16T00:39:59.371119+0000","last_fullsized":"2024-09-16T00:39:59.371119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373198+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:39:59.373198+0000","last_peered":"2024-09-16T00:39:59.373198+0000","last_clean":"2024-09-16T00:39:59.373198+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:39:59.373198+0000","last_undegraded":"2024-09-16T00:39:59.373198+0000","last_fullsized":"2024-09-16T00:39:59.373198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.594058+0000","last_change":"2024-09-16T00:39:59.593568+0000","last_active":"2024-09-16T00:39:59.594058+0000","last_peered":"2024-09-16T00:39:59.594058+0000","last_clean":"2024-09-16T00:39:59.594058+0000","last_became_active":"2024-09-16T00:39:59.593209+0000","last_became_peered":"2024-09-16T00:39:59.593209+0000","last_unstale":"2024-09-16T00:39:59.594058+0000","last_undegraded":"2024-09-16T00:39:59.594058+0000","last_fullsized":"2024-09-16T00:39:59.594058+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:56.516773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371238+0000","last_change":"2024-09-16T00:38:52.974330+0000","last_active":"2024-09-16T00:39:59.371238+0000","last_peered":"2024-09-16T00:39:59.371238+0000","last_clean":"2024-09-16T00:39:59.371238+0000","last_became_active":"2024-09-16T00:38:52.974004+0000","last_became_peered":"2024-09-16T00:38:52.974004+0000","last_unstale":"2024-09-16T00:39:59.371238+0000","last_undegraded":"2024-09-16T00:39:59.371238+0000","last_fullsized":"2024-09-16T00:39:59.371238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:31.820720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371467+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:39:59.371467+0000","last_peered":"2024-09-16T00:39:59.371467+0000","last_clean":"2024-09-16T00:39:59.371467+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:39:59.371467+0000","last_undegraded":"2024-09-16T00:39:59.371467+0000","last_fullsized":"2024-09-16T00:39:59.371467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.380258+0000","last_change":"2024-09-16T00:39:59.380258+0000","last_active":"2024-09-16T00:39:59.380258+0000","last_peered":"2024-09-16T00:39:59.380258+0000","last_clean":"2024-09-16T00:39:59.380258+0000","last_became_active":"2024-09-16T00:39:59.379813+0000","last_became_peered":"2024-09-16T00:39:59.379813+0000","last_unstale":"2024-09-16T00:39:59.380258+0000","last_undegraded":"2024-09-16T00:39:59.380258+0000","last_fullsized":"2024-09-16T00:39:59.380258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:03.110305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377545+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:39:59.377545+0000","last_peered":"2024-09-16T00:39:59.377545+0000","last_clean":"2024-09-16T00:39:59.377545+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:39:59.377545+0000","last_undegraded":"2024-09-16T00:39:59.377545+0000","last_fullsized":"2024-09-16T00:39:59.377545+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381185+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:39:59.381185+0000","last_peered":"2024-09-16T00:39:59.381185+0000","last_clean":"2024-09-16T00:39:59.381185+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:39:59.381185+0000","last_undegraded":"2024-09-16T00:39:59.381185+0000","last_fullsized":"2024-09-16T00:39:59.381185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371884+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:39:59.371884+0000","last_peered":"2024-09-16T00:39:59.371884+0000","last_clean":"2024-09-16T00:39:59.371884+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:39:59.371884+0000","last_undegraded":"2024-09-16T00:39:59.371884+0000","last_fullsized":"2024-09-16T00:39:59.371884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374306+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:39:59.374306+0000","last_peered":"2024-09-16T00:39:59.374306+0000","last_clean":"2024-09-16T00:39:59.374306+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:39:59.374306+0000","last_undegraded":"2024-09-16T00:39:59.374306+0000","last_fullsized":"2024-09-16T00:39:59.374306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":175,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:40:25.225224+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:40:25.225224+0000","last_peered":"2024-09-16T00:40:25.225224+0000","last_clean":"2024-09-16T00:40:25.225224+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:40:25.225224+0000","last_undegraded":"2024-09-16T00:40:25.225224+0000","last_fullsized":"2024-09-16T00:40:25.225224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63511,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":111,"num_read_kb":117,"num_write":30,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373436+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:39:59.373436+0000","last_peered":"2024-09-16T00:39:59.373436+0000","last_clean":"2024-09-16T00:39:59.373436+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:39:59.373436+0000","last_undegraded":"2024-09-16T00:39:59.373436+0000","last_fullsized":"2024-09-16T00:39:59.373436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589639+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:39:59.589639+0000","last_peered":"2024-09-16T00:39:59.589639+0000","last_clean":"2024-09-16T00:39:59.589639+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:39:59.589639+0000","last_undegraded":"2024-09-16T00:39:59.589639+0000","last_fullsized":"2024-09-16T00:39:59.589639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.372001+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:39:59.372001+0000","last_peered":"2024-09-16T00:39:59.372001+0000","last_clean":"2024-09-16T00:39:59.372001+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:39:59.372001+0000","last_undegraded":"2024-09-16T00:39:59.372001+0000","last_fullsized":"2024-09-16T00:39:59.372001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373321+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:39:59.373321+0000","last_peered":"2024-09-16T00:39:59.373321+0000","last_clean":"2024-09-16T00:39:59.373321+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:39:59.373321+0000","last_undegraded":"2024-09-16T00:39:59.373321+0000","last_fullsized":"2024-09-16T00:39:59.373321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.592089+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:39:59.592089+0000","last_peered":"2024-09-16T00:39:59.592089+0000","last_clean":"2024-09-16T00:39:59.592089+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:39:59.592089+0000","last_undegraded":"2024-09-16T00:39:59.592089+0000","last_fullsized":"2024-09-16T00:39:59.592089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371515+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:39:59.371515+0000","last_peered":"2024-09-16T00:39:59.371515+0000","last_clean":"2024-09-16T00:39:59.371515+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:39:59.371515+0000","last_undegraded":"2024-09-16T00:39:59.371515+0000","last_fullsized":"2024-09-16T00:39:59.371515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.372652+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:39:59.372652+0000","last_peered":"2024-09-16T00:39:59.372652+0000","last_clean":"2024-09-16T00:39:59.372652+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:39:59.372652+0000","last_undegraded":"2024-09-16T00:39:59.372652+0000","last_fullsized":"2024-09-16T00:39:59.372652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589601+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:39:59.589601+0000","last_peered":"2024-09-16T00:39:59.589601+0000","last_clean":"2024-09-16T00:39:59.589601+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:39:59.589601+0000","last_undegraded":"2024-09-16T00:39:59.589601+0000","last_fullsized":"2024-09-16T00:39:59.589601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590236+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:39:59.590236+0000","last_peered":"2024-09-16T00:39:59.590236+0000","last_clean":"2024-09-16T00:39:59.590236+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:39:59.590236+0000","last_undegraded":"2024-09-16T00:39:59.590236+0000","last_fullsized":"2024-09-16T00:39:59.590236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383032+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:39:59.383032+0000","last_peered":"2024-09-16T00:39:59.383032+0000","last_clean":"2024-09-16T00:39:59.383032+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:39:59.383032+0000","last_undegraded":"2024-09-16T00:39:59.383032+0000","last_fullsized":"2024-09-16T00:39:59.383032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590750+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:39:59.590750+0000","last_peered":"2024-09-16T00:39:59.590750+0000","last_clean":"2024-09-16T00:39:59.590750+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:39:59.590750+0000","last_undegraded":"2024-09-16T00:39:59.590750+0000","last_fullsized":"2024-09-16T00:39:59.590750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373135+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:39:59.373135+0000","last_peered":"2024-09-16T00:39:59.373135+0000","last_clean":"2024-09-16T00:39:59.373135+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:39:59.373135+0000","last_undegraded":"2024-09-16T00:39:59.373135+0000","last_fullsized":"2024-09-16T00:39:59.373135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371503+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:39:59.371503+0000","last_peered":"2024-09-16T00:39:59.371503+0000","last_clean":"2024-09-16T00:39:59.371503+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:39:59.371503+0000","last_undegraded":"2024-09-16T00:39:59.371503+0000","last_fullsized":"2024-09-16T00:39:59.371503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374701+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:39:59.374701+0000","last_peered":"2024-09-16T00:39:59.374701+0000","last_clean":"2024-09-16T00:39:59.374701+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:39:59.374701+0000","last_undegraded":"2024-09-16T00:39:59.374701+0000","last_fullsized":"2024-09-16T00:39:59.374701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381770+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:39:59.381770+0000","last_peered":"2024-09-16T00:39:59.381770+0000","last_clean":"2024-09-16T00:39:59.381770+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:39:59.381770+0000","last_undegraded":"2024-09-16T00:39:59.381770+0000","last_fullsized":"2024-09-16T00:39:59.381770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377157+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:39:59.377157+0000","last_peered":"2024-09-16T00:39:59.377157+0000","last_clean":"2024-09-16T00:39:59.377157+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:39:59.377157+0000","last_undegraded":"2024-09-16T00:39:59.377157+0000","last_fullsized":"2024-09-16T00:39:59.377157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589251+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:39:59.589251+0000","last_peered":"2024-09-16T00:39:59.589251+0000","last_clean":"2024-09-16T00:39:59.589251+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:39:59.589251+0000","last_undegraded":"2024-09-16T00:39:59.589251+0000","last_fullsized":"2024-09-16T00:39:59.589251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371611+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:39:59.371611+0000","last_peered":"2024-09-16T00:39:59.371611+0000","last_clean":"2024-09-16T00:39:59.371611+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:39:59.371611+0000","last_undegraded":"2024-09-16T00:39:59.371611+0000","last_fullsized":"2024-09-16T00:39:59.371611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381316+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:39:59.381316+0000","last_peered":"2024-09-16T00:39:59.381316+0000","last_clean":"2024-09-16T00:39:59.381316+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:39:59.381316+0000","last_undegraded":"2024-09-16T00:39:59.381316+0000","last_fullsized":"2024-09-16T00:39:59.381316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371704+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:39:59.371704+0000","last_peered":"2024-09-16T00:39:59.371704+0000","last_clean":"2024-09-16T00:39:59.371704+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:39:59.371704+0000","last_undegraded":"2024-09-16T00:39:59.371704+0000","last_fullsized":"2024-09-16T00:39:59.371704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373771+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:39:59.373771+0000","last_peered":"2024-09-16T00:39:59.373771+0000","last_clean":"2024-09-16T00:39:59.373771+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:39:59.373771+0000","last_undegraded":"2024-09-16T00:39:59.373771+0000","last_fullsized":"2024-09-16T00:39:59.373771+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381720+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:39:59.381720+0000","last_peered":"2024-09-16T00:39:59.381720+0000","last_clean":"2024-09-16T00:39:59.381720+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:39:59.381720+0000","last_undegraded":"2024-09-16T00:39:59.381720+0000","last_fullsized":"2024-09-16T00:39:59.381720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373893+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:39:59.373893+0000","last_peered":"2024-09-16T00:39:59.373893+0000","last_clean":"2024-09-16T00:39:59.373893+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:39:59.373893+0000","last_undegraded":"2024-09-16T00:39:59.373893+0000","last_fullsized":"2024-09-16T00:39:59.373893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.380959+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:39:59.380959+0000","last_peered":"2024-09-16T00:39:59.380959+0000","last_clean":"2024-09-16T00:39:59.380959+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:39:59.380959+0000","last_undegraded":"2024-09-16T00:39:59.380959+0000","last_fullsized":"2024-09-16T00:39:59.380959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381103+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:39:59.381103+0000","last_peered":"2024-09-16T00:39:59.381103+0000","last_clean":"2024-09-16T00:39:59.381103+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:39:59.381103+0000","last_undegraded":"2024-09-16T00:39:59.381103+0000","last_fullsized":"2024-09-16T00:39:59.381103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590367+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:39:59.590367+0000","last_peered":"2024-09-16T00:39:59.590367+0000","last_clean":"2024-09-16T00:39:59.590367+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:39:59.590367+0000","last_undegraded":"2024-09-16T00:39:59.590367+0000","last_fullsized":"2024-09-16T00:39:59.590367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.378308+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:39:59.378308+0000","last_peered":"2024-09-16T00:39:59.378308+0000","last_clean":"2024-09-16T00:39:59.378308+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:39:59.378308+0000","last_undegraded":"2024-09-16T00:39:59.378308+0000","last_fullsized":"2024-09-16T00:39:59.378308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374557+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:39:59.374557+0000","last_peered":"2024-09-16T00:39:59.374557+0000","last_clean":"2024-09-16T00:39:59.374557+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:39:59.374557+0000","last_undegraded":"2024-09-16T00:39:59.374557+0000","last_fullsized":"2024-09-16T00:39:59.374557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381810+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:39:59.381810+0000","last_peered":"2024-09-16T00:39:59.381810+0000","last_clean":"2024-09-16T00:39:59.381810+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:39:59.381810+0000","last_undegraded":"2024-09-16T00:39:59.381810+0000","last_fullsized":"2024-09-16T00:39:59.381810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.383559+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:39:59.383559+0000","last_peered":"2024-09-16T00:39:59.383559+0000","last_clean":"2024-09-16T00:39:59.383559+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:39:59.383559+0000","last_undegraded":"2024-09-16T00:39:59.383559+0000","last_fullsized":"2024-09-16T00:39:59.383559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371269+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:39:59.371269+0000","last_peered":"2024-09-16T00:39:59.371269+0000","last_clean":"2024-09-16T00:39:59.371269+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:39:59.371269+0000","last_undegraded":"2024-09-16T00:39:59.371269+0000","last_fullsized":"2024-09-16T00:39:59.371269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.382780+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:39:59.382780+0000","last_peered":"2024-09-16T00:39:59.382780+0000","last_clean":"2024-09-16T00:39:59.382780+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:39:59.382780+0000","last_undegraded":"2024-09-16T00:39:59.382780+0000","last_fullsized":"2024-09-16T00:39:59.382780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589955+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:39:59.589955+0000","last_peered":"2024-09-16T00:39:59.589955+0000","last_clean":"2024-09-16T00:39:59.589955+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:39:59.589955+0000","last_undegraded":"2024-09-16T00:39:59.589955+0000","last_fullsized":"2024-09-16T00:39:59.589955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.378063+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:39:59.378063+0000","last_peered":"2024-09-16T00:39:59.378063+0000","last_clean":"2024-09-16T00:39:59.378063+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:39:59.378063+0000","last_undegraded":"2024-09-16T00:39:59.378063+0000","last_fullsized":"2024-09-16T00:39:59.378063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373932+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:39:59.373932+0000","last_peered":"2024-09-16T00:39:59.373932+0000","last_clean":"2024-09-16T00:39:59.373932+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:39:59.373932+0000","last_undegraded":"2024-09-16T00:39:59.373932+0000","last_fullsized":"2024-09-16T00:39:59.373932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381657+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:39:59.381657+0000","last_peered":"2024-09-16T00:39:59.381657+0000","last_clean":"2024-09-16T00:39:59.381657+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:39:59.381657+0000","last_undegraded":"2024-09-16T00:39:59.381657+0000","last_fullsized":"2024-09-16T00:39:59.381657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374260+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:39:59.374260+0000","last_peered":"2024-09-16T00:39:59.374260+0000","last_clean":"2024-09-16T00:39:59.374260+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:39:59.374260+0000","last_undegraded":"2024-09-16T00:39:59.374260+0000","last_fullsized":"2024-09-16T00:39:59.374260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374983+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:39:59.374983+0000","last_peered":"2024-09-16T00:39:59.374983+0000","last_clean":"2024-09-16T00:39:59.374983+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:39:59.374983+0000","last_undegraded":"2024-09-16T00:39:59.374983+0000","last_fullsized":"2024-09-16T00:39:59.374983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589031+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:39:59.589031+0000","last_peered":"2024-09-16T00:39:59.589031+0000","last_clean":"2024-09-16T00:39:59.589031+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:39:59.589031+0000","last_undegraded":"2024-09-16T00:39:59.589031+0000","last_fullsized":"2024-09-16T00:39:59.589031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589750+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:39:59.589750+0000","last_peered":"2024-09-16T00:39:59.589750+0000","last_clean":"2024-09-16T00:39:59.589750+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:39:59.589750+0000","last_undegraded":"2024-09-16T00:39:59.589750+0000","last_fullsized":"2024-09-16T00:39:59.589750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589987+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:39:59.589987+0000","last_peered":"2024-09-16T00:39:59.589987+0000","last_clean":"2024-09-16T00:39:59.589987+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:39:59.589987+0000","last_undegraded":"2024-09-16T00:39:59.589987+0000","last_fullsized":"2024-09-16T00:39:59.589987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.377978+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:39:59.377978+0000","last_peered":"2024-09-16T00:39:59.377978+0000","last_clean":"2024-09-16T00:39:59.377978+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:39:59.377978+0000","last_undegraded":"2024-09-16T00:39:59.377978+0000","last_fullsized":"2024-09-16T00:39:59.377978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374112+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:39:59.374112+0000","last_peered":"2024-09-16T00:39:59.374112+0000","last_clean":"2024-09-16T00:39:59.374112+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:39:59.374112+0000","last_undegraded":"2024-09-16T00:39:59.374112+0000","last_fullsized":"2024-09-16T00:39:59.374112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371544+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:39:59.371544+0000","last_peered":"2024-09-16T00:39:59.371544+0000","last_clean":"2024-09-16T00:39:59.371544+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:39:59.371544+0000","last_undegraded":"2024-09-16T00:39:59.371544+0000","last_fullsized":"2024-09-16T00:39:59.371544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590122+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:39:59.590122+0000","last_peered":"2024-09-16T00:39:59.590122+0000","last_clean":"2024-09-16T00:39:59.590122+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:39:59.590122+0000","last_undegraded":"2024-09-16T00:39:59.590122+0000","last_fullsized":"2024-09-16T00:39:59.590122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589500+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:39:59.589500+0000","last_peered":"2024-09-16T00:39:59.589500+0000","last_clean":"2024-09-16T00:39:59.589500+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:39:59.589500+0000","last_undegraded":"2024-09-16T00:39:59.589500+0000","last_fullsized":"2024-09-16T00:39:59.589500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.381674+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:39:59.381674+0000","last_peered":"2024-09-16T00:39:59.381674+0000","last_clean":"2024-09-16T00:39:59.381674+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:39:59.381674+0000","last_undegraded":"2024-09-16T00:39:59.381674+0000","last_fullsized":"2024-09-16T00:39:59.381674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.371065+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:39:59.371065+0000","last_peered":"2024-09-16T00:39:59.371065+0000","last_clean":"2024-09-16T00:39:59.371065+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:39:59.371065+0000","last_undegraded":"2024-09-16T00:39:59.371065+0000","last_fullsized":"2024-09-16T00:39:59.371065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.589133+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:39:59.589133+0000","last_peered":"2024-09-16T00:39:59.589133+0000","last_clean":"2024-09-16T00:39:59.589133+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:39:59.589133+0000","last_undegraded":"2024-09-16T00:39:59.589133+0000","last_fullsized":"2024-09-16T00:39:59.589133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.373580+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:39:59.373580+0000","last_peered":"2024-09-16T00:39:59.373580+0000","last_clean":"2024-09-16T00:39:59.373580+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:39:59.373580+0000","last_undegraded":"2024-09-16T00:39:59.373580+0000","last_fullsized":"2024-09-16T00:39:59.373580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374059+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:39:59.374059+0000","last_peered":"2024-09-16T00:39:59.374059+0000","last_clean":"2024-09-16T00:39:59.374059+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:39:59.374059+0000","last_undegraded":"2024-09-16T00:39:59.374059+0000","last_fullsized":"2024-09-16T00:39:59.374059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590548+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:39:59.590548+0000","last_peered":"2024-09-16T00:39:59.590548+0000","last_clean":"2024-09-16T00:39:59.590548+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:39:59.590548+0000","last_undegraded":"2024-09-16T00:39:59.590548+0000","last_fullsized":"2024-09-16T00:39:59.590548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.590849+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:39:59.590849+0000","last_peered":"2024-09-16T00:39:59.590849+0000","last_clean":"2024-09-16T00:39:59.590849+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:39:59.590849+0000","last_undegraded":"2024-09-16T00:39:59.590849+0000","last_fullsized":"2024-09-16T00:39:59.590849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T00:39:59.374940+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:39:59.374940+0000","last_peered":"2024-09-16T00:39:59.374940+0000","last_clean":"2024-09-16T00:39:59.374940+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:39:59.374940+0000","last_undegraded":"2024-09-16T00:39:59.374940+0000","last_fullsized":"2024-09-16T00:39:59.374940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64309,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":117,"num_read_kb":123,"num_write":85,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":192927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547233,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1003520,"data_stored":499483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36992,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708160,"statfs":{"total":95995035648,"available":95957155840,"internally_reserved":0,"allocated":1683456,"data_stored":1153556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939691,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":500065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168625,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":500087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1605632,"data_stored":1094529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":495387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1609728,"data_stored":1090033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":499505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":495491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444111,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1077248,"data_stored":558932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1073152,"data_stored":558898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1003520,"data_stored":495477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:40:31.710 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:40:31.710 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T00:40:31.710 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T00:40:31.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T00:40:31.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:31 smithi081 ceph-mon[35539]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:31.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:31 smithi102 ceph-mon[35364]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:32.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.450+0000 7efc60ce8640 1 -- 172.21.15.31:0/3870005177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102570 msgr2=0x7efc5c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:32.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.450+0000 7efc60ce8640 1 --2- 172.21.15.31:0/3870005177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102570 0x7efc5c1029b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009900 tx=0x7efc4c02a480 comp rx=0 tx=0).stop 2024-09-16T00:40:32.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.450+0000 7efc60ce8640 1 -- 172.21.15.31:0/3870005177 shutdown_connections 2024-09-16T00:40:32.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.450+0000 7efc60ce8640 1 --2- 172.21.15.31:0/3870005177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc5c102ef0 0x7efc5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.450+0000 7efc60ce8640 1 --2- 172.21.15.31:0/3870005177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102570 0x7efc5c1029b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.450+0000 7efc60ce8640 1 --2- 172.21.15.31:0/3870005177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc5c100e20 0x7efc5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.450+0000 7efc60ce8640 1 -- 172.21.15.31:0/3870005177 >> 172.21.15.31:0/3870005177 conn(0x7efc5c0fc7d0 msgr2=0x7efc5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.450+0000 7efc60ce8640 1 -- 172.21.15.31:0/3870005177 shutdown_connections 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.450+0000 7efc60ce8640 1 -- 172.21.15.31:0/3870005177 wait complete. 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc60ce8640 1 Processor -- start 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc60ce8640 1 -- start start 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc60ce8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc5c100e20 0x7efc5c1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc60ce8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc5c102570 0x7efc5c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc60ce8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102ef0 0x7efc5c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc60ce8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c110fd0 con 0x7efc5c102ef0 2024-09-16T00:40:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc60ce8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c110e50 con 0x7efc5c102570 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc60ce8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c111150 con 0x7efc5c100e20 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc5ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102ef0 0x7efc5c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc5ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102ef0 0x7efc5c19bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49838/0 (socket says 172.21.15.31:49838) 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.451+0000 7efc5ad9d640 1 -- 172.21.15.31:0/2951274618 learned_addr learned my addr 172.21.15.31:0/2951274618 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc59d9b640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc5c102570 0x7efc5c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc5a59c640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc5c100e20 0x7efc5c1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc5ad9d640 1 -- 172.21.15.31:0/2951274618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc5c100e20 msgr2=0x7efc5c1973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc5ad9d640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc5c100e20 0x7efc5c1973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc5ad9d640 1 -- 172.21.15.31:0/2951274618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc5c102570 msgr2=0x7efc5c197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc5ad9d640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc5c102570 0x7efc5c197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc5ad9d640 1 -- 172.21.15.31:0/2951274618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c19c380 con 0x7efc5c102ef0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc5a59c640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc5c100e20 0x7efc5c1973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc59d9b640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc5c102570 0x7efc5c197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc5ad9d640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102ef0 0x7efc5c19bca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efc5400eb10 tx=0x7efc5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc537fe640 1 -- 172.21.15.31:0/2951274618 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54004b20 con 0x7efc5c102ef0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c19c670 con 0x7efc5c102ef0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc537fe640 1 -- 172.21.15.31:0/2951274618 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc54004cc0 con 0x7efc5c102ef0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc537fe640 1 -- 172.21.15.31:0/2951274618 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54010760 con 0x7efc5c102ef0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.452+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c1a3f50 con 0x7efc5c102ef0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.453+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5c101200 con 0x7efc5c102ef0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.454+0000 7efc537fe640 1 -- 172.21.15.31:0/2951274618 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efc54005000 con 0x7efc5c102ef0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.455+0000 7efc537fe640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efc480a32d0 0x7efc480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.455+0000 7efc5a59c640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efc480a32d0 0x7efc480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:32.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.455+0000 7efc537fe640 1 -- 172.21.15.31:0/2951274618 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7efc540d0940 con 0x7efc5c102ef0 2024-09-16T00:40:32.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.456+0000 7efc5a59c640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efc480a32d0 0x7efc480a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc44009b40 tx=0x7efc44005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:32.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.458+0000 7efc537fe640 1 -- 172.21.15.31:0/2951274618 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc5409d1d0 con 0x7efc5c102ef0 2024-09-16T00:40:32.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:32 smithi031 ceph-mon[30956]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:40:32.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.672+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efc5c1a4180 con 0x7efc5c102ef0 2024-09-16T00:40:32.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.672+0000 7efc537fe640 1 -- 172.21.15.31:0/2951274618 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7efc54085e20 con 0x7efc5c102ef0 2024-09-16T00:40:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:32.673 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T00:40:32.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efc480a32d0 msgr2=0x7efc480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:32.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efc480a32d0 0x7efc480a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc44009b40 tx=0x7efc44005e50 comp rx=0 tx=0).stop 2024-09-16T00:40:32.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102ef0 msgr2=0x7efc5c19bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:32.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102ef0 0x7efc5c19bca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efc5400eb10 tx=0x7efc5400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 shutdown_connections 2024-09-16T00:40:32.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c102ef0 0x7efc5c19bca0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc5c102570 0x7efc5c197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efc480a32d0 0x7efc480a5790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 --2- 172.21.15.31:0/2951274618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc5c100e20 0x7efc5c1973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:32.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 >> 172.21.15.31:0/2951274618 conn(0x7efc5c0fc7d0 msgr2=0x7efc5c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:32.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 shutdown_connections 2024-09-16T00:40:32.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:32.674+0000 7efc60ce8640 1 -- 172.21.15.31:0/2951274618 wait complete. 2024-09-16T00:40:32.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:32 smithi081 ceph-mon[35539]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:40:32.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:32 smithi102 ceph-mon[35364]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:40:33.365 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T00:40:33.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:33.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:33 smithi031 ceph-mon[30956]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:33.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:33 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2951274618' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:40:33.628 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:33 smithi081 ceph-mon[35539]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:33 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2951274618' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:40:33.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:33 smithi102 ceph-mon[35364]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:33.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:33 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2951274618' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.125+0000 7f37489c7640 1 -- 172.21.15.31:0/3290898032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744106f10 msgr2=0x7f374410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.125+0000 7f37489c7640 1 --2- 172.21.15.31:0/3290898032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744106f10 0x7f374410ea90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f373c009900 tx=0x7f373c02a410 comp rx=0 tx=0).stop 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.126+0000 7f37489c7640 1 -- 172.21.15.31:0/3290898032 shutdown_connections 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.126+0000 7f37489c7640 1 --2- 172.21.15.31:0/3290898032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744106f10 0x7f374410ea90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.126+0000 7f37489c7640 1 --2- 172.21.15.31:0/3290898032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3744102790 0x7f37441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.126+0000 7f37489c7640 1 --2- 172.21.15.31:0/3290898032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3744101de0 0x7f37441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.126+0000 7f37489c7640 1 -- 172.21.15.31:0/3290898032 >> 172.21.15.31:0/3290898032 conn(0x7f37440fd8c0 msgr2=0x7f37440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.126+0000 7f37489c7640 1 -- 172.21.15.31:0/3290898032 shutdown_connections 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.126+0000 7f37489c7640 1 -- 172.21.15.31:0/3290898032 wait complete. 2024-09-16T00:40:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.126+0000 7f37489c7640 1 Processor -- start 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.126+0000 7f37489c7640 1 -- start start 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f37489c7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3744101de0 0x7f3744198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f37489c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744102790 0x7f37441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f37489c7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3744106f10 0x7f374419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f37489c7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3744111f90 con 0x7f3744102790 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f37489c7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3744111e10 con 0x7f3744106f10 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f37489c7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3744112110 con 0x7f3744101de0 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744102790 0x7f37441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744102790 0x7f37441989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49870/0 (socket says 172.21.15.31:49870) 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3741d9b640 1 -- 172.21.15.31:0/958381010 learned_addr learned my addr 172.21.15.31:0/958381010 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f374259c640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3744101de0 0x7f3744198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3742d9d640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3744106f10 0x7f374419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3741d9b640 1 -- 172.21.15.31:0/958381010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3744101de0 msgr2=0x7f3744198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3741d9b640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3744101de0 0x7f3744198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3741d9b640 1 -- 172.21.15.31:0/958381010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3744106f10 msgr2=0x7f374419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3741d9b640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3744106f10 0x7f374419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3741d9b640 1 -- 172.21.15.31:0/958381010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374419d420 con 0x7f3744102790 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f3741d9b640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744102790 0x7f37441989b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f372c00b4a0 tx=0x7f372c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f373b7fe640 1 -- 172.21.15.31:0/958381010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c014070 con 0x7f3744102790 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374419d710 con 0x7f3744102790 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f373b7fe640 1 -- 172.21.15.31:0/958381010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372c010af0 con 0x7f3744102790 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.127+0000 7f373b7fe640 1 -- 172.21.15.31:0/958381010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c017d10 con 0x7f3744102790 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.128+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37441a4ff0 con 0x7f3744102790 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.129+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37441021c0 con 0x7f3744102790 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.130+0000 7f373b7fe640 1 -- 172.21.15.31:0/958381010 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f372c017070 con 0x7f3744102790 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.130+0000 7f373b7fe640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37300a32d0 0x7f37300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.131+0000 7f374259c640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37300a32d0 0x7f37300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.131+0000 7f373b7fe640 1 -- 172.21.15.31:0/958381010 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f372c0cf090 con 0x7f3744102790 2024-09-16T00:40:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.131+0000 7f374259c640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37300a32d0 0x7f37300a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3734005fd0 tx=0x7f3734009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.134+0000 7f373b7fe640 1 -- 172.21.15.31:0/958381010 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f372c09ba30 con 0x7f3744102790 2024-09-16T00:40:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.345+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f374402d7c0 con 0x7f3744102790 2024-09-16T00:40:34.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.346+0000 7f373b7fe640 1 -- 172.21.15.31:0/958381010 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f372c09b330 con 0x7f3744102790 2024-09-16T00:40:34.347 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:34.347 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:40:34.347 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:40:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.348+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37300a32d0 msgr2=0x7f37300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.348+0000 7f37489c7640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37300a32d0 0x7f37300a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3734005fd0 tx=0x7f3734009680 comp rx=0 tx=0).stop 2024-09-16T00:40:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.348+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744102790 msgr2=0x7f37441989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.348+0000 7f37489c7640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744102790 0x7f37441989b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f372c00b4a0 tx=0x7f372c00b960 comp rx=0 tx=0).stop 2024-09-16T00:40:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.349+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 shutdown_connections 2024-09-16T00:40:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.349+0000 7f37489c7640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3744106f10 0x7f374419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.349+0000 7f37489c7640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3744102790 0x7f37441989b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.349+0000 7f37489c7640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37300a32d0 0x7f37300a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.349+0000 7f37489c7640 1 --2- 172.21.15.31:0/958381010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3744101de0 0x7f3744198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.349+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 >> 172.21.15.31:0/958381010 conn(0x7f37440fd8c0 msgr2=0x7f37440fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.349+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 shutdown_connections 2024-09-16T00:40:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:34.349+0000 7f37489c7640 1 -- 172.21.15.31:0/958381010 wait complete. 2024-09-16T00:40:34.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:34 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/958381010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:34.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:34 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/958381010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:34.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:34 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/958381010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:35.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:35 smithi081 ceph-mon[35539]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:35.795 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:35 smithi031 ceph-mon[30956]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:35 smithi102 ceph-mon[35364]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.876+0000 7fa154cc6640 1 -- 172.21.15.31:0/1928656302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 msgr2=0x7fa15010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.876+0000 7fa154cc6640 1 --2- 172.21.15.31:0/1928656302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 0x7fa15010ea90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa140009ca0 tx=0x7fa14002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.877+0000 7fa154cc6640 1 -- 172.21.15.31:0/1928656302 shutdown_connections 2024-09-16T00:40:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.877+0000 7fa154cc6640 1 --2- 172.21.15.31:0/1928656302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 0x7fa15010ea90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.877+0000 7fa154cc6640 1 --2- 172.21.15.31:0/1928656302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150102790 0x7fa1501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.877+0000 7fa154cc6640 1 --2- 172.21.15.31:0/1928656302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa150101de0 0x7fa1501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.877+0000 7fa154cc6640 1 -- 172.21.15.31:0/1928656302 >> 172.21.15.31:0/1928656302 conn(0x7fa1500fd8c0 msgr2=0x7fa1500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.877+0000 7fa154cc6640 1 -- 172.21.15.31:0/1928656302 shutdown_connections 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.877+0000 7fa154cc6640 1 -- 172.21.15.31:0/1928656302 wait complete. 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.877+0000 7fa154cc6640 1 Processor -- start 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.877+0000 7fa154cc6640 1 -- start start 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa154cc6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150101de0 0x7fa1501985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa154cc6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa150102790 0x7fa150198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa154cc6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 0x7fa15019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa154cc6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa150112010 con 0x7fa150106f10 2024-09-16T00:40:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa154cc6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa150111e90 con 0x7fa150102790 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa154cc6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa150112190 con 0x7fa150101de0 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 0x7fa15019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 0x7fa15019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49878/0 (socket says 172.21.15.31:49878) 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14ed9d640 1 -- 172.21.15.31:0/2597113777 learned_addr learned my addr 172.21.15.31:0/2597113777 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14e59c640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150101de0 0x7fa1501985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14dd9b640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa150102790 0x7fa150198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14ed9d640 1 -- 172.21.15.31:0/2597113777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150101de0 msgr2=0x7fa1501985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14ed9d640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150101de0 0x7fa1501985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14ed9d640 1 -- 172.21.15.31:0/2597113777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa150102790 msgr2=0x7fa150198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14ed9d640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa150102790 0x7fa150198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14ed9d640 1 -- 172.21.15.31:0/2597113777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15019d5a0 con 0x7fa150106f10 2024-09-16T00:40:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa14ed9d640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 0x7fa15019cec0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa140031a60 tx=0x7fa140031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa1477fe640 1 -- 172.21.15.31:0/2597113777 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140003b70 con 0x7fa150106f10 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.878+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15019d830 con 0x7fa150106f10 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.879+0000 7fa1477fe640 1 -- 172.21.15.31:0/2597113777 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa140003d10 con 0x7fa150106f10 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.879+0000 7fa1477fe640 1 -- 172.21.15.31:0/2597113777 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140004770 con 0x7fa150106f10 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.879+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1501a5170 con 0x7fa150106f10 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.880+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa150102240 con 0x7fa150106f10 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.881+0000 7fa1477fe640 1 -- 172.21.15.31:0/2597113777 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa14002f790 con 0x7fa150106f10 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.882+0000 7fa1477fe640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa13c0a32d0 0x7fa13c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.882+0000 7fa1477fe640 1 -- 172.21.15.31:0/2597113777 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fa1400f1140 con 0x7fa150106f10 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.882+0000 7fa14e59c640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa13c0a32d0 0x7fa13c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.882+0000 7fa14e59c640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa13c0a32d0 0x7fa13c0a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa148009990 tx=0x7fa148009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:35.885+0000 7fa1477fe640 1 -- 172.21.15.31:0/2597113777 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1400bda80 con 0x7fa150106f10 2024-09-16T00:40:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.097+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa15010a120 con 0x7fa150106f10 2024-09-16T00:40:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.097+0000 7fa1477fe640 1 -- 172.21.15.31:0/2597113777 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fa1400bd380 con 0x7fa150106f10 2024-09-16T00:40:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:36.098 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:40:36.098 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.099+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa13c0a32d0 msgr2=0x7fa13c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.099+0000 7fa154cc6640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa13c0a32d0 0x7fa13c0a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa148009990 tx=0x7fa148009110 comp rx=0 tx=0).stop 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 msgr2=0x7fa15019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 0x7fa15019cec0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa140031a60 tx=0x7fa140031a90 comp rx=0 tx=0).stop 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 shutdown_connections 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106f10 0x7fa15019cec0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa150102790 0x7fa150198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa13c0a32d0 0x7fa13c0a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 --2- 172.21.15.31:0/2597113777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150101de0 0x7fa1501985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 >> 172.21.15.31:0/2597113777 conn(0x7fa1500fd8c0 msgr2=0x7fa1500fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 shutdown_connections 2024-09-16T00:40:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:36.100+0000 7fa154cc6640 1 -- 172.21.15.31:0/2597113777 wait complete. 2024-09-16T00:40:36.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:36 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2597113777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:36.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:36 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2597113777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:36.817 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:36 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2597113777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:36.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:40:37.114 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:37.496 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:37 smithi031 ceph-mon[30956]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.579+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3475857595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc106f10 msgr2=0x7fb3cc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.579+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3475857595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc106f10 0x7fb3cc10ea90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009ca0 tx=0x7fb3c402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.579+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3475857595 shutdown_connections 2024-09-16T00:40:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.579+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3475857595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc106f10 0x7fb3cc10ea90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.579+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3475857595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3cc102790 0x7fb3cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.579+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3475857595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3cc101de0 0x7fb3cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.579+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3475857595 >> 172.21.15.31:0/3475857595 conn(0x7fb3cc0fd8c0 msgr2=0x7fb3cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.579+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3475857595 shutdown_connections 2024-09-16T00:40:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.579+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3475857595 wait complete. 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d3ca4640 1 Processor -- start 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d3ca4640 1 -- start start 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d3ca4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3cc101de0 0x7fb3cc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d3ca4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc102790 0x7fb3cc198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d3ca4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3cc106f10 0x7fb3cc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d3ca4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc111fc0 con 0x7fb3cc102790 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d3ca4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc111e40 con 0x7fb3cc101de0 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d3ca4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc112140 con 0x7fb3cc106f10 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d123f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc102790 0x7fb3cc198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d123f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc102790 0x7fb3cc198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58630/0 (socket says 172.21.15.31:58630) 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d123f640 1 -- 172.21.15.31:0/3045078328 learned_addr learned my addr 172.21.15.31:0/3045078328 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d1a40640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3cc101de0 0x7fb3cc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d123f640 1 -- 172.21.15.31:0/3045078328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3cc106f10 msgr2=0x7fb3cc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d123f640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3cc106f10 0x7fb3cc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d123f640 1 -- 172.21.15.31:0/3045078328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3cc101de0 msgr2=0x7fb3cc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d123f640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3cc101de0 0x7fb3cc1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d123f640 1 -- 172.21.15.31:0/3045078328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3cc19d290 con 0x7fb3cc102790 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d1a40640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3cc101de0 0x7fb3cc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.580+0000 7fb3d123f640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc102790 0x7fb3cc198820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00c920 tx=0x7fb3bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.581+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/3045078328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc007c70 con 0x7fb3cc102790 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.581+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/3045078328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3bc007e10 con 0x7fb3cc102790 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.581+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3cc19d580 con 0x7fb3cc102790 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.581+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/3045078328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc018890 con 0x7fb3cc102790 2024-09-16T00:40:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.581+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3cc1a4e60 con 0x7fb3cc102790 2024-09-16T00:40:37.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.582+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3cc1021c0 con 0x7fb3cc102790 2024-09-16T00:40:37.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.583+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/3045078328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb3bc018a50 con 0x7fb3cc102790 2024-09-16T00:40:37.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.584+0000 7fb3c2ffd640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3b80a3350 0x7fb3b80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:37.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.584+0000 7fb3d1a40640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3b80a3350 0x7fb3b80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:37.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.584+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/3045078328 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fb3bc0d0160 con 0x7fb3cc102790 2024-09-16T00:40:37.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.584+0000 7fb3d1a40640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3b80a3350 0x7fb3b80a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800baa0 tx=0x7fb3c8005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:37.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.587+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/3045078328 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3bc014030 con 0x7fb3cc102790 2024-09-16T00:40:37.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:37 smithi081 ceph-mon[35539]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:37.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.765+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3cc1a5200 con 0x7fb3cc102790 2024-09-16T00:40:37.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.767+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/3045078328 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24561 (secure 0 0 0) 0x7fb3bc01f300 con 0x7fb3cc102790 2024-09-16T00:40:37.767 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:37.767 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:39:59.366751+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:40:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3b80a3350 msgr2=0x7fb3b80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3b80a3350 0x7fb3b80a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800baa0 tx=0x7fb3c8005d90 comp rx=0 tx=0).stop 2024-09-16T00:40:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc102790 msgr2=0x7fb3cc198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc102790 0x7fb3cc198820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00c920 tx=0x7fb3bc00cde0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 shutdown_connections 2024-09-16T00:40:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb3b80a3350 0x7fb3b80a5810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3cc106f10 0x7fb3cc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3cc102790 0x7fb3cc198820 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 --2- 172.21.15.31:0/3045078328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3cc101de0 0x7fb3cc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 >> 172.21.15.31:0/3045078328 conn(0x7fb3cc0fd8c0 msgr2=0x7fb3cc0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 shutdown_connections 2024-09-16T00:40:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:37.769+0000 7fb3d3ca4640 1 -- 172.21.15.31:0/3045078328 wait complete. 2024-09-16T00:40:37.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:37 smithi102 ceph-mon[35364]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:38.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:38 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3045078328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:40:38.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:38.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:38 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3045078328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:40:38.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:38 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3045078328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:40:38.800 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.386+0000 7f34a0861640 1 -- 172.21.15.31:0/2437348030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 msgr2=0x7f349c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.386+0000 7f34a0861640 1 --2- 172.21.15.31:0/2437348030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 0x7f349c1029b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3484009900 tx=0x7f348402a480 comp rx=0 tx=0).stop 2024-09-16T00:40:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.386+0000 7f34a0861640 1 -- 172.21.15.31:0/2437348030 shutdown_connections 2024-09-16T00:40:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.386+0000 7f34a0861640 1 --2- 172.21.15.31:0/2437348030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f349c102ef0 0x7f349c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.386+0000 7f34a0861640 1 --2- 172.21.15.31:0/2437348030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 0x7f349c1029b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.386+0000 7f34a0861640 1 --2- 172.21.15.31:0/2437348030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f349c100e20 0x7f349c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.386+0000 7f34a0861640 1 -- 172.21.15.31:0/2437348030 >> 172.21.15.31:0/2437348030 conn(0x7f349c0fc7d0 msgr2=0x7f349c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.386+0000 7f34a0861640 1 -- 172.21.15.31:0/2437348030 shutdown_connections 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.386+0000 7f34a0861640 1 -- 172.21.15.31:0/2437348030 wait complete. 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f34a0861640 1 Processor -- start 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f34a0861640 1 -- start start 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f34a0861640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f349c100e20 0x7f349c197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f34a0861640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 0x7f349c197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f34a0861640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f349c102ef0 0x7f349c19c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f34a0861640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f349c110f80 con 0x7f349c102570 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f34a0861640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f349c110e00 con 0x7f349c100e20 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f34a0861640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f349c111100 con 0x7f349c102ef0 2024-09-16T00:40:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f3493fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 0x7f349c197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f3493fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 0x7f349c197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58640/0 (socket says 172.21.15.31:58640) 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f3493fff640 1 -- 172.21.15.31:0/2620665054 learned_addr learned my addr 172.21.15.31:0/2620665054 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f349ad9d640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f349c102ef0 0x7f349c19c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f349a59c640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f349c100e20 0x7f349c197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f3493fff640 1 -- 172.21.15.31:0/2620665054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f349c102ef0 msgr2=0x7f349c19c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f3493fff640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f349c102ef0 0x7f349c19c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f3493fff640 1 -- 172.21.15.31:0/2620665054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f349c100e20 msgr2=0x7f349c197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f3493fff640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f349c100e20 0x7f349c197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f3493fff640 1 -- 172.21.15.31:0/2620665054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349c19c5d0 con 0x7f349c102570 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.387+0000 7f349a59c640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f349c100e20 0x7f349c197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.388+0000 7f3493fff640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 0x7f349c197580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3484006010 tx=0x7f3484003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.388+0000 7f34937fe640 1 -- 172.21.15.31:0/2620665054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348403f070 con 0x7f349c102570 2024-09-16T00:40:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.388+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349c19c860 con 0x7f349c102570 2024-09-16T00:40:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.388+0000 7f34937fe640 1 -- 172.21.15.31:0/2620665054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3484031c30 con 0x7f349c102570 2024-09-16T00:40:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.388+0000 7f34937fe640 1 -- 172.21.15.31:0/2620665054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484035030 con 0x7f349c102570 2024-09-16T00:40:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.388+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349c19cd90 con 0x7f349c102570 2024-09-16T00:40:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.389+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f349c1021b0 con 0x7f349c102570 2024-09-16T00:40:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.391+0000 7f34937fe640 1 -- 172.21.15.31:0/2620665054 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3484042070 con 0x7f349c102570 2024-09-16T00:40:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.392+0000 7f34937fe640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34880a32d0 0x7f34880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.392+0000 7f349a59c640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34880a32d0 0x7f34880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.392+0000 7f34937fe640 1 -- 172.21.15.31:0/2620665054 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f34840f0be0 con 0x7f349c102570 2024-09-16T00:40:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.392+0000 7f349a59c640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34880a32d0 0x7f34880a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f348c005fd0 tx=0x7f348c00b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:39.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.395+0000 7f34937fe640 1 -- 172.21.15.31:0/2620665054 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3484031de0 con 0x7f349c102570 2024-09-16T00:40:39.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.608+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f349c101200 con 0x7f349c102570 2024-09-16T00:40:39.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.609+0000 7f34937fe640 1 -- 172.21.15.31:0/2620665054 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f349c101200 con 0x7f349c102570 2024-09-16T00:40:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:39.609 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:40:39.610 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:40:39.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34880a32d0 msgr2=0x7f34880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:39.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34880a32d0 0x7f34880a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f348c005fd0 tx=0x7f348c00b5f0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 msgr2=0x7f349c197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:39.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 0x7f349c197580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3484006010 tx=0x7f3484003c10 comp rx=0 tx=0).stop 2024-09-16T00:40:39.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 shutdown_connections 2024-09-16T00:40:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34880a32d0 0x7f34880a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f349c102ef0 0x7f349c19c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f349c102570 0x7f349c197580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 --2- 172.21.15.31:0/2620665054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f349c100e20 0x7f349c197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 >> 172.21.15.31:0/2620665054 conn(0x7f349c0fc7d0 msgr2=0x7f349c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 shutdown_connections 2024-09-16T00:40:39.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:39.611+0000 7f34a0861640 1 -- 172.21.15.31:0/2620665054 wait complete. 2024-09-16T00:40:39.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:39 smithi081 ceph-mon[35539]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:39.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:39 smithi102 ceph-mon[35364]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:39.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:39 smithi031 ceph-mon[30956]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:40.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:38:53.576781+0000', 'modified': '2024-09-16T00:39:35.954166+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6839/3021718993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3021718993}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3021718993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6839/2243721734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 2243721734}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 2243721734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482]} 2024-09-16T00:40:40.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T00:40:40.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:40:40.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:40 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2620665054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:40.642 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:40.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:40 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2620665054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:40.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:40 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2620665054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:41.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.129+0000 7f02fc04d640 1 -- 172.21.15.31:0/1345700005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4102d40 msgr2=0x7f02f4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.129+0000 7f02fc04d640 1 --2- 172.21.15.31:0/1345700005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4102d40 0x7f02f4103120 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f002a480 comp rx=0 tx=0).stop 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.129+0000 7f02fc04d640 1 -- 172.21.15.31:0/1345700005 shutdown_connections 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.129+0000 7f02fc04d640 1 --2- 172.21.15.31:0/1345700005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f410c170 0x7f02f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.129+0000 7f02fc04d640 1 --2- 172.21.15.31:0/1345700005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4103660 0x7f02f410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.129+0000 7f02fc04d640 1 --2- 172.21.15.31:0/1345700005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4102d40 0x7f02f4103120 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.129+0000 7f02fc04d640 1 -- 172.21.15.31:0/1345700005 >> 172.21.15.31:0/1345700005 conn(0x7f02f40fc7a0 msgr2=0x7f02f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.129+0000 7f02fc04d640 1 -- 172.21.15.31:0/1345700005 shutdown_connections 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.129+0000 7f02fc04d640 1 -- 172.21.15.31:0/1345700005 wait complete. 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02fc04d640 1 Processor -- start 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02fc04d640 1 -- start start 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02fc04d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4102d40 0x7f02f4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02fc04d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4103660 0x7f02f4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02fc04d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f410c170 0x7f02f419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02fc04d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4111f70 con 0x7f02f4103660 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02fc04d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4111df0 con 0x7f02f410c170 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02fc04d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41120f0 con 0x7f02f4102d40 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02f95e8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4103660 0x7f02f4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02f95e8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4103660 0x7f02f4198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58664/0 (socket says 172.21.15.31:58664) 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.130+0000 7f02f95e8640 1 -- 172.21.15.31:0/692028535 learned_addr learned my addr 172.21.15.31:0/692028535 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f410c170 0x7f02f419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02f9de9640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4102d40 0x7f02f4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02f95e8640 1 -- 172.21.15.31:0/692028535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4102d40 msgr2=0x7f02f4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02f95e8640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4102d40 0x7f02f4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02f95e8640 1 -- 172.21.15.31:0/692028535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f410c170 msgr2=0x7f02f419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02f95e8640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f410c170 0x7f02f419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02f95e8640 1 -- 172.21.15.31:0/692028535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f419d300 con 0x7f02f4103660 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f410c170 0x7f02f419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02f95e8640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4103660 0x7f02f4198890 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f02e400c970 tx=0x7f02e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02eaffd640 1 -- 172.21.15.31:0/692028535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4007bb0 con 0x7f02f4103660 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02eaffd640 1 -- 172.21.15.31:0/692028535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4007d50 con 0x7f02f4103660 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02fc04d640 1 -- 172.21.15.31:0/692028535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f419d5f0 con 0x7f02f4103660 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02eaffd640 1 -- 172.21.15.31:0/692028535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4018820 con 0x7f02f4103660 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.131+0000 7f02fc04d640 1 -- 172.21.15.31:0/692028535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f41a4ed0 con 0x7f02f4103660 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02eaffd640 1 -- 172.21.15.31:0/692028535 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f02e40189c0 con 0x7f02f4103660 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02fe426740 1 Processor -- start 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02fe426740 1 -- start start 2024-09-16T00:40:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02fe426740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 0x56122e480c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02fe426740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e48fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02fe426740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e4923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02fe426740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56122e3e7090 con 0x56122e48ffc0 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02fe426740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56122e38ea70 con 0x56122e4811d0 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02fe426740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56122e3e3050 con 0x56122e4808b0 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.134+0000 7f02eaffd640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f02e00a32d0 0x7f02e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02f95e8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e4923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02fa5ea640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 0x56122e480c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02fa5ea640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 0x56122e480c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:42008/0 (socket says 172.21.15.31:42008) 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02fa5ea640 1 -- 172.21.15.31:0/820806820 learned_addr learned my addr 172.21.15.31:0/820806820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02f9de9640 1 --2- 172.21.15.31:0/820806820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e48fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02eaffd640 1 -- 172.21.15.31:0/692028535 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f02e40cffa0 con 0x7f02f4103660 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f02e00a32d0 0x7f02e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02f95e8640 1 -- 172.21.15.31:0/820806820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 msgr2=0x56122e480c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02f95e8640 1 --2- 172.21.15.31:0/820806820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 0x56122e480c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02f95e8640 1 -- 172.21.15.31:0/820806820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 msgr2=0x56122e48fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02f95e8640 1 --2- 172.21.15.31:0/820806820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e48fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02f95e8640 1 -- 172.21.15.31:0/820806820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122e39b130 con 0x56122e48ffc0 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02f95e8640 1 --2- 172.21.15.31:0/820806820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e4923b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f02e40cd5c0 tx=0x7f02e406ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edf1fa619b3c823 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f02e00a32d0 0x7f02e00a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f02ec013c40 tx=0x7f02ec0072a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02dffff640 1 -- 172.21.15.31:0/820806820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e409c7d0 con 0x56122e48ffc0 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02dffff640 1 -- 172.21.15.31:0/820806820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4066f70 con 0x56122e48ffc0 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.135+0000 7f02dffff640 1 -- 172.21.15.31:0/820806820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e400fac0 con 0x56122e48ffc0 2024-09-16T00:40:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.136+0000 7f02fe426740 1 -- 172.21.15.31:0/820806820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 msgr2=0x56122e4923b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.136+0000 7f02fe426740 1 --2- 172.21.15.31:0/820806820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e4923b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f02e40cd5c0 tx=0x7f02e406ef50 comp rx=0 tx=0).stop 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.136+0000 7f02fe426740 1 -- 172.21.15.31:0/820806820 shutdown_connections 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.136+0000 7f02fe426740 1 --2- 172.21.15.31:0/820806820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e4923b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.136+0000 7f02fe426740 1 --2- 172.21.15.31:0/820806820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e48fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.136+0000 7f02fe426740 1 --2- 172.21.15.31:0/820806820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 0x56122e480c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.136+0000 7f02fe426740 1 -- 172.21.15.31:0/820806820 >> 172.21.15.31:0/820806820 conn(0x56122e47cf60 msgr2=0x56122e47d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.136+0000 7f02fe426740 1 -- 172.21.15.31:0/820806820 shutdown_connections 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.136+0000 7f02fe426740 1 -- 172.21.15.31:0/820806820 wait complete. 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fe426740 1 -- start start 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fe426740 1 Processor -- start 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fe426740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 0x56122e48f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fe426740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e48cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fe426740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e48d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fe426740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56122e492b90 con 0x56122e48ffc0 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fe426740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56122e3a1f20 con 0x56122e4811d0 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fe426740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56122e492d10 con 0x56122e4808b0 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fe426740 10 client.0 _ll_register_callbacks cb 0x56122e36b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fa5ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e48d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fa5ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e48d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58692/0 (socket says 172.21.15.31:58692) 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fa5ea640 1 -- 172.21.15.31:0/327166368 learned_addr learned my addr 172.21.15.31:0/327166368 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02f9de9640 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 0x56122e48f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02f95e8640 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e48cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fa5ea640 1 -- 172.21.15.31:0/327166368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 msgr2=0x56122e48f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 0x56122e48f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fa5ea640 1 -- 172.21.15.31:0/327166368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 msgr2=0x56122e48cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e48cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02fa5ea640 1 -- 172.21.15.31:0/327166368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122e1582b0 con 0x56122e48ffc0 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.137+0000 7f02f95e8640 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e48cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e48d110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0135c0 tx=0x7f02ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b47982be592db21 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02dd7fa640 1 -- 172.21.15.31:0/327166368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec024070 con 0x56122e48ffc0 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02fe426740 1 -- 172.21.15.31:0/327166368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e4066f70 con 0x56122e48ffc0 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02dd7fa640 1 -- 172.21.15.31:0/327166368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f02ec00e840 con 0x56122e48ffc0 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02dd7fa640 1 -- 172.21.15.31:0/327166368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f02ec030050 con 0x56122e48ffc0 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02fe426740 10 client.15045 fetch_fsmap learned FSMap version 20 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02fe426740 1 -- 172.21.15.31:0/327166368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56122e49e580 con 0x56122e48ffc0 2024-09-16T00:40:41.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02dd7fa640 1 -- 172.21.15.31:0/327166368 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4066f70 con 0x56122e48ffc0 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02dd7fa640 1 -- 172.21.15.31:0/327166368 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f02ec020a40 con 0x56122e48ffc0 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.138+0000 7f02fe426740 10 client.15045 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.139+0000 7f02fe426740 10 client.15045 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.139+0000 7f02fe426740 20 client.15045 populate_metadata read hostname 'smithi031' 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.139+0000 7f02fe426740 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56122e4a0bc0 0x56122e4a2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.139+0000 7f02fe426740 4 client.15045 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.139+0000 7f02fe426740 1 -- 172.21.15.31:0/327166368 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56122e3f4600 con 0x56122e4a0bc0 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.139+0000 7f02f9de9640 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56122e4a0bc0 0x56122e4a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.139+0000 7f02f9de9640 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56122e4a0bc0 0x56122e4a2030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21dbdb014a9f7751 server_cookie=69aa73d2448ba555 in_seq=0 out_seq=0 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.139+0000 7f02dd7fa640 10 client.15045 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02dd7fa640 1 -- 172.21.15.31:0/327166368 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56122e3f4600 con 0x56122e4a0bc0 2024-09-16T00:40:41.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02dd7fa640 10 client.15045 handle_command_reply: tid=0 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 client.15045 shutdown 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 -- 172.21.15.31:0/327166368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 msgr2=0x56122e48d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e48d110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0135c0 tx=0x7f02ec004ab0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 -- 172.21.15.31:0/327166368 shutdown_connections 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e48ffc0 0x56122e48d110 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e48cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56122e4a0bc0 0x56122e4a2030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 --2- 172.21.15.31:0/327166368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4808b0 0x56122e48f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02dd7fa640 0 client.15045 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 -- 172.21.15.31:0/327166368 >> 172.21.15.31:0/327166368 conn(0x56122e47cf60 msgr2=0x56122e482cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 -- 172.21.15.31:0/327166368 shutdown_connections 2024-09-16T00:40:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 1 -- 172.21.15.31:0/327166368 wait complete. 2024-09-16T00:40:41.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.140+0000 7f02fe426740 20 client.15045 trim_cache size 0 max 16384 2024-09-16T00:40:41.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.158+0000 7f02fe426740 1 Processor -- start 2024-09-16T00:40:41.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.158+0000 7f02fe426740 1 -- start start 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.158+0000 7f02fe426740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4a2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.158+0000 7f02fe426740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4811d0 0x56122e48b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.158+0000 7f02fe426740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e48ffc0 0x56122e48b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.158+0000 7f02fe426740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56122e4a0370 con 0x56122e4808b0 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.158+0000 7f02fe426740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56122e4a0a40 con 0x56122e48ffc0 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.158+0000 7f02fe426740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56122e4a1110 con 0x56122e4811d0 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f9de9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f9de9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4a2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58704/0 (socket says 172.21.15.31:58704) 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f9de9640 1 -- 172.21.15.31:0/836867800 learned_addr learned my addr 172.21.15.31:0/836867800 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f95e8640 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4811d0 0x56122e48b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e48ffc0 0x56122e48b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f9de9640 1 -- 172.21.15.31:0/836867800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4811d0 msgr2=0x56122e48b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f9de9640 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4811d0 0x56122e48b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f9de9640 1 -- 172.21.15.31:0/836867800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e48ffc0 msgr2=0x56122e48b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f9de9640 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e48ffc0 0x56122e48b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f9de9640 1 -- 172.21.15.31:0/836867800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122e48b120 con 0x56122e4808b0 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f95e8640 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4811d0 0x56122e48b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e48ffc0 0x56122e48b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02f9de9640 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4a2030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db8b5a17c588219 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02df7fe640 1 -- 172.21.15.31:0/836867800 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0049070 con 0x56122e4808b0 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02df7fe640 1 -- 172.21.15.31:0/836867800 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f0004240 con 0x56122e4808b0 2024-09-16T00:40:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.159+0000 7f02df7fe640 1 -- 172.21.15.31:0/836867800 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0035040 con 0x56122e4808b0 2024-09-16T00:40:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 -- 172.21.15.31:0/836867800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 msgr2=0x56122e4a2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4a2030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f0005a20 comp rx=0 tx=0).stop 2024-09-16T00:40:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 -- 172.21.15.31:0/836867800 shutdown_connections 2024-09-16T00:40:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e48ffc0 0x56122e48b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e4811d0 0x56122e48b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 --2- 172.21.15.31:0/836867800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4a2030 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 -- 172.21.15.31:0/836867800 >> 172.21.15.31:0/836867800 conn(0x56122e47cf60 msgr2=0x56122e48cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 -- 172.21.15.31:0/836867800 shutdown_connections 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 -- 172.21.15.31:0/836867800 wait complete. 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.160+0000 7f02fe426740 1 -- start start 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fe426740 1 Processor -- start 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fe426740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fe426740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e4aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fe426740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e48ffc0 0x56122e4a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fe426740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56122e4a1eb0 con 0x56122e4808b0 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fe426740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56122e4a17e0 con 0x56122e4811d0 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fe426740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56122e4a2580 con 0x56122e48ffc0 2024-09-16T00:40:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fe426740 10 client.0 _ll_register_callbacks cb 0x7f02ec0134f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02f9de9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02f9de9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4aa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58712/0 (socket says 172.21.15.31:58712) 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02f9de9640 1 -- 172.21.15.31:0/2763592492 learned_addr learned my addr 172.21.15.31:0/2763592492 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02f9de9640 1 -- 172.21.15.31:0/2763592492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e48ffc0 msgr2=0x56122e4a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e48ffc0 0x56122e4a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02f95e8640 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e4aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02f9de9640 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e48ffc0 0x56122e4a3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02f9de9640 1 -- 172.21.15.31:0/2763592492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 msgr2=0x56122e4aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02f9de9640 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e4aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02f9de9640 1 -- 172.21.15.31:0/2763592492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122e4a3f80 con 0x56122e4808b0 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.161+0000 7f02fa5ea640 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e48ffc0 0x56122e4a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02f9de9640 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4aa440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f02f002a4c0 tx=0x7f02f0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af6cfa397994eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02dd7fa640 1 -- 172.21.15.31:0/2763592492 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0035050 con 0x56122e4808b0 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02dd7fa640 1 -- 172.21.15.31:0/2763592492 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f02f0005220 con 0x56122e4808b0 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02fe426740 1 -- 172.21.15.31:0/2763592492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f0004240 con 0x56122e4808b0 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02dd7fa640 1 -- 172.21.15.31:0/2763592492 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f02f005f050 con 0x56122e4808b0 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02fe426740 10 client.15051 fetch_fsmap learned FSMap version 20 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02fe426740 1 -- 172.21.15.31:0/2763592492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56122e4a8970 con 0x56122e4808b0 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02dd7fa640 1 -- 172.21.15.31:0/2763592492 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f0004240 con 0x56122e4808b0 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02dd7fa640 1 -- 172.21.15.31:0/2763592492 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f02f00037b0 con 0x56122e4808b0 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.162+0000 7f02fe426740 10 client.15051 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.163+0000 7f02fe426740 10 client.15051 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:40:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.163+0000 7f02fe426740 20 client.15051 populate_metadata read hostname 'smithi031' 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.163+0000 7f02fe426740 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56122e50f3c0 0x56122e50e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.163+0000 7f02fe426740 4 client.15051 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.163+0000 7f02fe426740 1 -- 172.21.15.31:0/2763592492 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "status"}) -- 0x56122e3eeb10 con 0x56122e50f3c0 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.163+0000 7f02f95e8640 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56122e50f3c0 0x56122e50e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.163+0000 7f02f95e8640 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56122e50f3c0 0x56122e50e0d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46a9581f15427f4 server_cookie=240dec610bfc9f52 in_seq=0 out_seq=0 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.163+0000 7f02dd7fa640 10 client.15051 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02dd7fa640 1 -- 172.21.15.31:0/2763592492 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56122e3eeb10 con 0x56122e50f3c0 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02dd7fa640 10 client.15051 handle_command_reply: tid=0 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 client.15051 shutdown 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 -- 172.21.15.31:0/2763592492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 msgr2=0x56122e4aa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4aa440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f02f002a4c0 tx=0x7f02f0004040 comp rx=0 tx=0).stop 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 -- 172.21.15.31:0/2763592492 shutdown_connections 2024-09-16T00:40:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56122e50f3c0 0x56122e50e0d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56122e48ffc0 0x56122e4a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56122e4811d0 0x56122e4aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 --2- 172.21.15.31:0/2763592492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56122e4808b0 0x56122e4aa440 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02dd7fa640 0 client.15051 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 -- 172.21.15.31:0/2763592492 >> 172.21.15.31:0/2763592492 conn(0x56122e47cf60 msgr2=0x56122e48b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 -- 172.21.15.31:0/2763592492 shutdown_connections 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 1 -- 172.21.15.31:0/2763592492 wait complete. 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fe426740 20 client.15051 trim_cache size 0 max 16384 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:40:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "ccb601ea-73c2-11ef-bceb-c7b262605968", 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24482, 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 107.15749216899999, 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 20, 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 118.10522893 2024-09-16T00:40:41.171 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:40:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fc04d640 1 -- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f02e00a32d0 msgr2=0x7f02e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.164+0000 7f02fc04d640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f02e00a32d0 0x7f02e00a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f02ec013c40 tx=0x7f02ec0072a0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 -- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4103660 msgr2=0x7f02f4198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4103660 0x7f02f4198890 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f02e400c970 tx=0x7f02e400ce30 comp rx=0 tx=0).stop 2024-09-16T00:40:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 -- 172.21.15.31:0/692028535 shutdown_connections 2024-09-16T00:40:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f410c170 0x7f02f419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02f4103660 0x7f02f4198890 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f02e00a32d0 0x7f02e00a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 --2- 172.21.15.31:0/692028535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4102d40 0x7f02f4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 -- 172.21.15.31:0/692028535 >> 172.21.15.31:0/692028535 conn(0x7f02f40fc7a0 msgr2=0x7f02f40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 -- 172.21.15.31:0/692028535 shutdown_connections 2024-09-16T00:40:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:41.165+0000 7f02fc04d640 1 -- 172.21.15.31:0/692028535 wait complete. 2024-09-16T00:40:41.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:41 smithi081 ceph-mon[35539]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:41.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:41 smithi102 ceph-mon[35364]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:41.805 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:41 smithi031 ceph-mon[30956]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:41.889 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T00:40:41.889 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:40:41.899 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:40:41.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T00:40:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.616+0000 7f375faad640 1 -- 172.21.15.31:0/3684503583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758102de0 msgr2=0x7f3758105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.616+0000 7f375faad640 1 --2- 172.21.15.31:0/3684503583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758102de0 0x7f3758105250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3748009900 tx=0x7f374802a410 comp rx=0 tx=0).stop 2024-09-16T00:40:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.616+0000 7f375faad640 1 -- 172.21.15.31:0/3684503583 shutdown_connections 2024-09-16T00:40:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.616+0000 7f375faad640 1 --2- 172.21.15.31:0/3684503583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3758105790 0x7f3758105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.616+0000 7f375faad640 1 --2- 172.21.15.31:0/3684503583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758102de0 0x7f3758105250 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.616+0000 7f375faad640 1 --2- 172.21.15.31:0/3684503583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3758100490 0x7f37581028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.616+0000 7f375faad640 1 -- 172.21.15.31:0/3684503583 >> 172.21.15.31:0/3684503583 conn(0x7f37580fc2a0 msgr2=0x7f37580fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.616+0000 7f375faad640 1 -- 172.21.15.31:0/3684503583 shutdown_connections 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.616+0000 7f375faad640 1 -- 172.21.15.31:0/3684503583 wait complete. 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.617+0000 7f375faad640 1 Processor -- start 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.617+0000 7f375faad640 1 -- start start 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.617+0000 7f375faad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758100490 0x7f375819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.617+0000 7f375faad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3758102de0 0x7f375819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.617+0000 7f375faad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3758105790 0x7f37581a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.617+0000 7f375faad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3758113ea0 con 0x7f3758100490 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.617+0000 7f375faad640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3758113d20 con 0x7f3758102de0 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.617+0000 7f375faad640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3758114020 con 0x7f3758105790 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d849640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758100490 0x7f375819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d849640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758100490 0x7f375819e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58734/0 (socket says 172.21.15.31:58734) 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d849640 1 -- 172.21.15.31:0/3542941735 learned_addr learned my addr 172.21.15.31:0/3542941735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375e04a640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3758105790 0x7f37581a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d048640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3758102de0 0x7f375819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d849640 1 -- 172.21.15.31:0/3542941735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3758105790 msgr2=0x7f37581a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d849640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3758105790 0x7f37581a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d849640 1 -- 172.21.15.31:0/3542941735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3758102de0 msgr2=0x7f375819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d849640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3758102de0 0x7f375819ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d849640 1 -- 172.21.15.31:0/3542941735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37581a6110 con 0x7f3758100490 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375d849640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758100490 0x7f375819e4e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f375400d950 tx=0x7f375400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f374effd640 1 -- 172.21.15.31:0/3542941735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754017030 con 0x7f3758100490 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f374effd640 1 -- 172.21.15.31:0/3542941735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3754004020 con 0x7f3758100490 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37581a6400 con 0x7f3758100490 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f374effd640 1 -- 172.21.15.31:0/3542941735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37540047a0 con 0x7f3758100490 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.618+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37581a6960 con 0x7f3758100490 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.619+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37580fdd90 con 0x7f3758100490 2024-09-16T00:40:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.620+0000 7f374effd640 1 -- 172.21.15.31:0/3542941735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f37540041d0 con 0x7f3758100490 2024-09-16T00:40:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.621+0000 7f374effd640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37440a3380 0x7f37440a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.621+0000 7f374effd640 1 -- 172.21.15.31:0/3542941735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f37540cfe10 con 0x7f3758100490 2024-09-16T00:40:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.621+0000 7f375d048640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37440a3380 0x7f37440a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.622+0000 7f375d048640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37440a3380 0x7f37440a5840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3748000c00 tx=0x7f37480315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.625+0000 7f374effd640 1 -- 172.21.15.31:0/3542941735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f375409c720 con 0x7f3758100490 2024-09-16T00:40:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.820+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3758062630 con 0x7f37440a3380 2024-09-16T00:40:42.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.821+0000 7f374effd640 1 -- 172.21.15.31:0/3542941735 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3758062630 con 0x7f37440a3380 2024-09-16T00:40:42.821 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-09-16T00:40:42.822 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-09-16T00:40:42.822 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-09-16T00:40:42.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.823+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37440a3380 msgr2=0x7f37440a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:42.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.823+0000 7f375faad640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37440a3380 0x7f37440a5840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3748000c00 tx=0x7f37480315d0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758100490 msgr2=0x7f375819e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:42.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758100490 0x7f375819e4e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f375400d950 tx=0x7f375400de10 comp rx=0 tx=0).stop 2024-09-16T00:40:42.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 shutdown_connections 2024-09-16T00:40:42.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37440a3380 0x7f37440a5840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3758105790 0x7f37581a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3758102de0 0x7f375819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 --2- 172.21.15.31:0/3542941735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3758100490 0x7f375819e4e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:42.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 >> 172.21.15.31:0/3542941735 conn(0x7f37580fc2a0 msgr2=0x7f3758100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:42.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 shutdown_connections 2024-09-16T00:40:42.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:42.824+0000 7f375faad640 1 -- 172.21.15.31:0/3542941735 wait complete. 2024-09-16T00:40:43.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T00:40:43.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:43 smithi081 ceph-mon[35539]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:43.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:43 smithi102 ceph-mon[35364]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:43.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:43 smithi031 ceph-mon[30956]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:44.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.373+0000 7fa477904640 1 -- 172.21.15.31:0/3408198211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 msgr2=0x7fa470102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:44.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.373+0000 7fa477904640 1 --2- 172.21.15.31:0/3408198211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 0x7fa470102000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009900 tx=0x7fa46c02a410 comp rx=0 tx=0).stop 2024-09-16T00:40:44.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.373+0000 7fa477904640 1 -- 172.21.15.31:0/3408198211 shutdown_connections 2024-09-16T00:40:44.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.373+0000 7fa477904640 1 --2- 172.21.15.31:0/3408198211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4701037c0 0x7fa47010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.373+0000 7fa477904640 1 --2- 172.21.15.31:0/3408198211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa470102e00 0x7fa470103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.373+0000 7fa477904640 1 --2- 172.21.15.31:0/3408198211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 0x7fa470102000 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.373+0000 7fa477904640 1 -- 172.21.15.31:0/3408198211 >> 172.21.15.31:0/3408198211 conn(0x7fa4700fd390 msgr2=0x7fa4700ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.373+0000 7fa477904640 1 -- 172.21.15.31:0/3408198211 shutdown_connections 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.373+0000 7fa477904640 1 -- 172.21.15.31:0/3408198211 wait complete. 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa477904640 1 Processor -- start 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa477904640 1 -- start start 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa477904640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 0x7fa470198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa477904640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa470102e00 0x7fa470198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa477904640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4701037c0 0x7fa47019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa477904640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010d960 con 0x7fa470101c00 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa477904640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010d7e0 con 0x7fa470102e00 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa477904640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010dae0 con 0x7fa4701037c0 2024-09-16T00:40:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa4756a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 0x7fa470198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa4756a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 0x7fa470198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58748/0 (socket says 172.21.15.31:58748) 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.374+0000 7fa4756a0640 1 -- 172.21.15.31:0/2801971284 learned_addr learned my addr 172.21.15.31:0/2801971284 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa474e9f640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa470102e00 0x7fa470198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa475ea1640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4701037c0 0x7fa47019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa4756a0640 1 -- 172.21.15.31:0/2801971284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4701037c0 msgr2=0x7fa47019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa4756a0640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4701037c0 0x7fa47019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa4756a0640 1 -- 172.21.15.31:0/2801971284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa470102e00 msgr2=0x7fa470198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa4756a0640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa470102e00 0x7fa470198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa4756a0640 1 -- 172.21.15.31:0/2801971284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47019fc80 con 0x7fa470101c00 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa474e9f640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa470102e00 0x7fa470198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa4756a0640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 0x7fa470198050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa46c031610 tx=0x7fa46c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa4667fc640 1 -- 172.21.15.31:0/2801971284 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c03f030 con 0x7fa470101c00 2024-09-16T00:40:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa4667fc640 1 -- 172.21.15.31:0/2801971284 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46c003ce0 con 0x7fa470101c00 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47019ff10 con 0x7fa470101c00 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa4667fc640 1 -- 172.21.15.31:0/2801971284 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c035070 con 0x7fa470101c00 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.375+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4701a0390 con 0x7fa470101c00 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.376+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4701090a0 con 0x7fa470101c00 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.377+0000 7fa4667fc640 1 -- 172.21.15.31:0/2801971284 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa46c042030 con 0x7fa470101c00 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.378+0000 7fa4667fc640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa45c0a3300 0x7fa45c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.378+0000 7fa474e9f640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa45c0a3300 0x7fa45c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.378+0000 7fa4667fc640 1 -- 172.21.15.31:0/2801971284 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fa46c0f0780 con 0x7fa470101c00 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.379+0000 7fa474e9f640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa45c0a3300 0x7fa45c0a57c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa470199570 tx=0x7fa460008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.381+0000 7fa4667fc640 1 -- 172.21.15.31:0/2801971284 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa46c0bcfb0 con 0x7fa470101c00 2024-09-16T00:40:44.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.565+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa470062630 con 0x7fa45c0a3300 2024-09-16T00:40:44.566 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:44 smithi031 ceph-mon[30956]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.578+0000 7fa4667fc640 1 -- 172.21.15.31:0/2801971284 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7fa470062630 con 0x7fa45c0a3300 2024-09-16T00:40:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:40:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:mds.a smithi031 running (2m) 52s ago 2m 15.8M - 19.1.1-326-g8921c8ef f1455bd0219c ce1c6b7e983d 2024-09-16T00:40:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:mds.b smithi081 running (2m) 54s ago 2m 26.4M - 19.1.1-326-g8921c8ef f1455bd0219c 00e3bc8e521c 2024-09-16T00:40:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:mds.c smithi102 running (2m) 54s ago 2m 14.4M - 19.1.1-326-g8921c8ef f1455bd0219c 95a1cc22f1c2 2024-09-16T00:40:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:mds.d smithi031 running (2m) 52s ago 2m 16.7M - 19.1.1-326-g8921c8ef f1455bd0219c 71e2abc19399 2024-09-16T00:40:44.578 INFO:teuthology.orchestra.run.smithi031.stdout:mds.e smithi081 running (2m) 54s ago 2m 14.1M - 19.1.1-326-g8921c8ef f1455bd0219c df6645692f33 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mds.f smithi102 running (2m) 54s ago 2m 16.6M - 19.1.1-326-g8921c8ef f1455bd0219c 99fe3278e5e6 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mds.g smithi031 running (2m) 52s ago 2m 14.4M - 19.1.1-326-g8921c8ef f1455bd0219c c8e9d3c8e852 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mds.h smithi081 running (2m) 54s ago 2m 13.6M - 19.1.1-326-g8921c8ef f1455bd0219c 3a4d39797acb 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mds.i smithi102 running (2m) 54s ago 2m 18.2M - 19.1.1-326-g8921c8ef f1455bd0219c 76b0ed73d828 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mds.j smithi031 running (2m) 52s ago 2m 14.4M - 19.1.1-326-g8921c8ef f1455bd0219c f856e32d6506 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mds.k smithi081 running (2m) 54s ago 2m 15.9M - 19.1.1-326-g8921c8ef f1455bd0219c 616001c7e847 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mds.l smithi102 running (2m) 54s ago 2m 14.5M - 19.1.1-326-g8921c8ef f1455bd0219c c73279967303 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.x smithi031 *:9283,8765 running (9m) 52s ago 9m 507M - 19.1.1-326-g8921c8ef f1455bd0219c 509e45728803 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.y smithi081 *:8443,8765 running (7m) 54s ago 7m 444M - 19.1.1-326-g8921c8ef f1455bd0219c 51d2485aecbb 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.z smithi102 *:8443,8765 running (7m) 54s ago 7m 441M - 19.1.1-326-g8921c8ef f1455bd0219c 2f053d2f087d 2024-09-16T00:40:44.579 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a smithi031 running (9m) 52s ago 9m 50.0M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 4b873356c0c1 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b smithi081 running (8m) 54s ago 8m 40.5M 2048M 19.1.1-326-g8921c8ef f1455bd0219c ad386f878d0a 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c smithi102 running (8m) 54s ago 8m 40.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c ebc8467ae9d6 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 52s ago 7m 54.0M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 39e78f2f169e 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi081 running (7m) 54s ago 7m 56.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c b4d2e77149af 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi102 running (6m) 54s ago 6m 52.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 3052100a338e 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi031 running (6m) 52s ago 6m 52.5M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 51f5087630ad 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi081 running (5m) 54s ago 5m 42.2M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 5a4a49fcf807 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi102 running (5m) 54s ago 5m 52.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 3d271fdffb31 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 smithi031 running (5m) 52s ago 5m 44.8M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 00388baf15ea 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 smithi081 running (4m) 54s ago 4m 45.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 7d38de5339ed 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 smithi102 running (4m) 54s ago 4m 44.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 7204089148a0 2024-09-16T00:40:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 smithi031 running (3m) 52s ago 3m 52.2M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 2a909b49bd19 2024-09-16T00:40:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 smithi081 running (3m) 54s ago 3m 51.3M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 315eb81b5a3c 2024-09-16T00:40:44.581 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 smithi102 running (2m) 54s ago 2m 49.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 5332fb60da34 2024-09-16T00:40:44.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa45c0a3300 msgr2=0x7fa45c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:44.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa45c0a3300 0x7fa45c0a57c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa470199570 tx=0x7fa460008040 comp rx=0 tx=0).stop 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 msgr2=0x7fa470198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 0x7fa470198050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa46c031610 tx=0x7fa46c0043b0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 shutdown_connections 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa45c0a3300 0x7fa45c0a57c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4701037c0 0x7fa47019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa470102e00 0x7fa470198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 --2- 172.21.15.31:0/2801971284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa470101c00 0x7fa470198050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 >> 172.21.15.31:0/2801971284 conn(0x7fa4700fd390 msgr2=0x7fa470106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 shutdown_connections 2024-09-16T00:40:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:44.580+0000 7fa477904640 1 -- 172.21.15.31:0/2801971284 wait complete. 2024-09-16T00:40:44.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:44 smithi102 ceph-mon[35364]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:44.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:44 smithi081 ceph-mon[35539]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:45.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T00:40:45.670 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:45.792 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:45 smithi031 ceph-mon[30956]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:45.792 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:45 smithi031 ceph-mon[30956]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:45.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:45 smithi102 ceph-mon[35364]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:45.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:45 smithi102 ceph-mon[35364]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:45.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:45 smithi081 ceph-mon[35539]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:45.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:45 smithi081 ceph-mon[35539]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.173+0000 7fa967222640 1 -- 172.21.15.31:0/2035880475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 msgr2=0x7fa960104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.173+0000 7fa967222640 1 --2- 172.21.15.31:0/2035880475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 0x7fa960104af0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa954009900 tx=0x7fa95402a410 comp rx=0 tx=0).stop 2024-09-16T00:40:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.173+0000 7fa967222640 1 -- 172.21.15.31:0/2035880475 shutdown_connections 2024-09-16T00:40:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.173+0000 7fa967222640 1 --2- 172.21.15.31:0/2035880475 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa960107960 0x7fa960109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.173+0000 7fa967222640 1 --2- 172.21.15.31:0/2035880475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa960105030 0x7fa960107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.173+0000 7fa967222640 1 --2- 172.21.15.31:0/2035880475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 0x7fa960104af0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.173+0000 7fa967222640 1 -- 172.21.15.31:0/2035880475 >> 172.21.15.31:0/2035880475 conn(0x7fa9600fc2a0 msgr2=0x7fa9600fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.173+0000 7fa967222640 1 -- 172.21.15.31:0/2035880475 shutdown_connections 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.173+0000 7fa967222640 1 -- 172.21.15.31:0/2035880475 wait complete. 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa967222640 1 Processor -- start 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa967222640 1 -- start start 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa967222640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 0x7fa960197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa967222640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa960105030 0x7fa960198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa967222640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa960107960 0x7fa96019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa967222640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010d8a0 con 0x7fa960102700 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa967222640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010d720 con 0x7fa960105030 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa967222640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010da20 con 0x7fa960107960 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa964fbe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 0x7fa960197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa964fbe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 0x7fa960197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58776/0 (socket says 172.21.15.31:58776) 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa964fbe640 1 -- 172.21.15.31:0/3617388495 learned_addr learned my addr 172.21.15.31:0/3617388495 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa95ffff640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa960105030 0x7fa960198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa9657bf640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa960107960 0x7fa96019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa964fbe640 1 -- 172.21.15.31:0/3617388495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa960107960 msgr2=0x7fa96019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa964fbe640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa960107960 0x7fa96019f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa964fbe640 1 -- 172.21.15.31:0/3617388495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa960105030 msgr2=0x7fa960198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa964fbe640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa960105030 0x7fa960198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa964fbe640 1 -- 172.21.15.31:0/3617388495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96019fb00 con 0x7fa960102700 2024-09-16T00:40:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.174+0000 7fa95ffff640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa960105030 0x7fa960198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.175+0000 7fa9657bf640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa960107960 0x7fa96019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.175+0000 7fa964fbe640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 0x7fa960197ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa954009230 tx=0x7fa954004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.175+0000 7fa95dffb640 1 -- 172.21.15.31:0/3617388495 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95403f030 con 0x7fa960102700 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.175+0000 7fa95dffb640 1 -- 172.21.15.31:0/3617388495 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa954003ce0 con 0x7fa960102700 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.175+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96019fd90 con 0x7fa960102700 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.175+0000 7fa95dffb640 1 -- 172.21.15.31:0/3617388495 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954035070 con 0x7fa960102700 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.175+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601a0200 con 0x7fa960102700 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.176+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9600fdd90 con 0x7fa960102700 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.177+0000 7fa95dffb640 1 -- 172.21.15.31:0/3617388495 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa954042030 con 0x7fa960102700 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.178+0000 7fa95dffb640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9500a3380 0x7fa9500a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.178+0000 7fa95ffff640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9500a3380 0x7fa9500a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.178+0000 7fa95dffb640 1 -- 172.21.15.31:0/3617388495 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fa9540f0220 con 0x7fa960102700 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.179+0000 7fa95ffff640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9500a3380 0x7fa9500a5840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa9601993f0 tx=0x7fa94c005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.183+0000 7fa95dffb640 1 -- 172.21.15.31:0/3617388495 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9540bca50 con 0x7fa960102700 2024-09-16T00:40:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.365+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fa960062630 con 0x7fa9500a3380 2024-09-16T00:40:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.371+0000 7fa95dffb640 1 -- 172.21.15.31:0/3617388495 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fa960062630 con 0x7fa9500a3380 2024-09-16T00:40:46.372 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T00:40:46.372 INFO:teuthology.orchestra.run.smithi031.stdout:mds.all 12/12 56s ago 2m smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-09-16T00:40:46.372 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 3/3 56s ago 7m smithi031=x;smithi081=y;smithi102=z;count:3 2024-09-16T00:40:46.372 INFO:teuthology.orchestra.run.smithi031.stdout:mon 3/3 56s ago 8m smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c;count:3 2024-09-16T00:40:46.372 INFO:teuthology.orchestra.run.smithi031.stdout:osd 12 56s ago - 2024-09-16T00:40:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9500a3380 msgr2=0x7fa9500a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9500a3380 0x7fa9500a5840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa9601993f0 tx=0x7fa94c005d90 comp rx=0 tx=0).stop 2024-09-16T00:40:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 msgr2=0x7fa960197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 0x7fa960197ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa954009230 tx=0x7fa954004430 comp rx=0 tx=0).stop 2024-09-16T00:40:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 shutdown_connections 2024-09-16T00:40:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9500a3380 0x7fa9500a5840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa960107960 0x7fa96019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa960105030 0x7fa960198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 --2- 172.21.15.31:0/3617388495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa960102700 0x7fa960197ed0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 >> 172.21.15.31:0/3617388495 conn(0x7fa9600fc2a0 msgr2=0x7fa960102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.373+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 shutdown_connections 2024-09-16T00:40:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:46.374+0000 7fa967222640 1 -- 172.21.15.31:0/3617388495 wait complete. 2024-09-16T00:40:47.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T00:40:47.311 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:47.732 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:47 smithi031 ceph-mon[30956]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:47.732 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:47 smithi031 ceph-mon[30956]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:47.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:47 smithi102 ceph-mon[35364]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:47.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:47 smithi102 ceph-mon[35364]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:47.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.815+0000 7fede6ec7640 1 -- 172.21.15.31:0/1469049282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0102660 msgr2=0x7fede0104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:47.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.815+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1469049282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0102660 0x7fede0104a50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009900 tx=0x7fedd002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:40:47.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.815+0000 7fede6ec7640 1 -- 172.21.15.31:0/1469049282 shutdown_connections 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.815+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1469049282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fede01078c0 0x7fede0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.815+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1469049282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0104f90 0x7fede0107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.815+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1469049282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0102660 0x7fede0104a50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.815+0000 7fede6ec7640 1 -- 172.21.15.31:0/1469049282 >> 172.21.15.31:0/1469049282 conn(0x7fede00fc280 msgr2=0x7fede00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 -- 172.21.15.31:0/1469049282 shutdown_connections 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 -- 172.21.15.31:0/1469049282 wait complete. 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 Processor -- start 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 -- start start 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0102660 0x7fede0101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0104f90 0x7fede0100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fede01078c0 0x7fede0100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fede010d890 con 0x7fede0104f90 2024-09-16T00:40:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fede010d710 con 0x7fede01078c0 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7fede6ec7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fede010da10 con 0x7fede0102660 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.816+0000 7feddffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0104f90 0x7fede0100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7feddffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0104f90 0x7fede0100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58802/0 (socket says 172.21.15.31:58802) 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7feddffff640 1 -- 172.21.15.31:0/1133734480 learned_addr learned my addr 172.21.15.31:0/1133734480 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7fede4c63640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0102660 0x7fede0101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7fede5464640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fede01078c0 0x7fede0100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7feddffff640 1 -- 172.21.15.31:0/1133734480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0102660 msgr2=0x7fede0101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7feddffff640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0102660 0x7fede0101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7feddffff640 1 -- 172.21.15.31:0/1133734480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fede01078c0 msgr2=0x7fede0100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7feddffff640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fede01078c0 0x7fede0100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7feddffff640 1 -- 172.21.15.31:0/1133734480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede0101410 con 0x7fede0104f90 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7feddffff640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0104f90 0x7fede0100610 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fedd400d740 tx=0x7fedd400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7fedddffb640 1 -- 172.21.15.31:0/1133734480 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4017070 con 0x7fede0104f90 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7fedddffb640 1 -- 172.21.15.31:0/1133734480 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd400de50 con 0x7fede0104f90 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7fedddffb640 1 -- 172.21.15.31:0/1133734480 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4004e00 con 0x7fede0104f90 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede01a1ff0 con 0x7fede0104f90 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.817+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede01a24a0 con 0x7fede0104f90 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.819+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede00fdd70 con 0x7fede0104f90 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.820+0000 7fedddffb640 1 -- 172.21.15.31:0/1133734480 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fedd4004fa0 con 0x7fede0104f90 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.821+0000 7fedddffb640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fedc80a3350 0x7fedc80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.821+0000 7fede4c63640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fedc80a3350 0x7fedc80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.821+0000 7fedddffb640 1 -- 172.21.15.31:0/1133734480 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fedd40cf710 con 0x7fede0104f90 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.821+0000 7fede4c63640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fedc80a3350 0x7fedc80a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fedd00314e0 tx=0x7fedd0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:47.825+0000 7fedddffb640 1 -- 172.21.15.31:0/1133734480 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fedd409bfa0 con 0x7fede0104f90 2024-09-16T00:40:47.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:47 smithi081 ceph-mon[35539]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:47.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:47 smithi081 ceph-mon[35539]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.009+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fede0062630 con 0x7fedc80a3350 2024-09-16T00:40:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.010+0000 7fedddffb640 1 -- 172.21.15.31:0/1133734480 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fede0062630 con 0x7fedc80a3350 2024-09-16T00:40:48.011 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-09-16T00:40:48.011 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-09-16T00:40:48.011 INFO:teuthology.orchestra.run.smithi031.stdout:smithi081 172.21.15.81 2024-09-16T00:40:48.011 INFO:teuthology.orchestra.run.smithi031.stdout:smithi102 172.21.15.102 2024-09-16T00:40:48.011 INFO:teuthology.orchestra.run.smithi031.stdout:3 hosts in cluster 2024-09-16T00:40:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fedc80a3350 msgr2=0x7fedc80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fedc80a3350 0x7fedc80a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fedd00314e0 tx=0x7fedd0000bc0 comp rx=0 tx=0).stop 2024-09-16T00:40:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0104f90 msgr2=0x7fede0100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0104f90 0x7fede0100610 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fedd400d740 tx=0x7fedd400dc00 comp rx=0 tx=0).stop 2024-09-16T00:40:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 shutdown_connections 2024-09-16T00:40:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fede01078c0 0x7fede0100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede0104f90 0x7fede0100610 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fedc80a3350 0x7fedc80a5810 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 --2- 172.21.15.31:0/1133734480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0102660 0x7fede0101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 >> 172.21.15.31:0/1133734480 conn(0x7fede00fc280 msgr2=0x7fede0102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 shutdown_connections 2024-09-16T00:40:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:48.013+0000 7fede6ec7640 1 -- 172.21.15.31:0/1133734480 wait complete. 2024-09-16T00:40:48.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T00:40:49.061 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.568+0000 7fc3f7a87640 1 -- 172.21.15.31:0/3553175537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f0101c00 msgr2=0x7fc3f0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.568+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/3553175537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f0101c00 0x7fc3f0102000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009900 tx=0x7fc3ec02a410 comp rx=0 tx=0).stop 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.568+0000 7fc3f7a87640 1 -- 172.21.15.31:0/3553175537 shutdown_connections 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.568+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/3553175537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3f01037c0 0x7fc3f010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.568+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/3553175537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f0102e00 0x7fc3f0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.568+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/3553175537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f0101c00 0x7fc3f0102000 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.568+0000 7fc3f7a87640 1 -- 172.21.15.31:0/3553175537 >> 172.21.15.31:0/3553175537 conn(0x7fc3f00fd390 msgr2=0x7fc3f00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.568+0000 7fc3f7a87640 1 -- 172.21.15.31:0/3553175537 shutdown_connections 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.568+0000 7fc3f7a87640 1 -- 172.21.15.31:0/3553175537 wait complete. 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f7a87640 1 Processor -- start 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f7a87640 1 -- start start 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f7a87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3f0101c00 0x7fc3f0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f7a87640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f0102e00 0x7fc3f00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f7a87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f01037c0 0x7fc3f0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f7a87640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f010d860 con 0x7fc3f01037c0 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f7a87640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f010d6e0 con 0x7fc3f0101c00 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f7a87640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f010d9e0 con 0x7fc3f0102e00 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f6024640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f01037c0 0x7fc3f0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f6024640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f01037c0 0x7fc3f0074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58822/0 (socket says 172.21.15.31:58822) 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f6024640 1 -- 172.21.15.31:0/1920584786 learned_addr learned my addr 172.21.15.31:0/1920584786 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f5823640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3f0101c00 0x7fc3f0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f5022640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f0102e00 0x7fc3f00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f6024640 1 -- 172.21.15.31:0/1920584786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f0102e00 msgr2=0x7fc3f00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f6024640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f0102e00 0x7fc3f00745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f6024640 1 -- 172.21.15.31:0/1920584786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3f0101c00 msgr2=0x7fc3f0075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f6024640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3f0101c00 0x7fc3f0075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f6024640 1 -- 172.21.15.31:0/1920584786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f00753e0 con 0x7fc3f01037c0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.569+0000 7fc3f5823640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3f0101c00 0x7fc3f0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.570+0000 7fc3f6024640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f01037c0 0x7fc3f0074b20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800cad0 tx=0x7fc3e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.570+0000 7fc3e6ffd640 1 -- 172.21.15.31:0/1920584786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800ae20 con 0x7fc3f01037c0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.570+0000 7fc3e6ffd640 1 -- 172.21.15.31:0/1920584786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e8005020 con 0x7fc3f01037c0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.570+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f01a1fe0 con 0x7fc3f01037c0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.570+0000 7fc3e6ffd640 1 -- 172.21.15.31:0/1920584786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800f6f0 con 0x7fc3f01037c0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.570+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f01a24a0 con 0x7fc3f01037c0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.571+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f0108fa0 con 0x7fc3f01037c0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.572+0000 7fc3e6ffd640 1 -- 172.21.15.31:0/1920584786 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc3e80051d0 con 0x7fc3f01037c0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.573+0000 7fc3e6ffd640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3dc0a32d0 0x7fc3dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.573+0000 7fc3e6ffd640 1 -- 172.21.15.31:0/1920584786 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fc3e80cfef0 con 0x7fc3f01037c0 2024-09-16T00:40:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.573+0000 7fc3f5823640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3dc0a32d0 0x7fc3dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:49.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.573+0000 7fc3f5823640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3dc0a32d0 0x7fc3dc0a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:49.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.577+0000 7fc3e6ffd640 1 -- 172.21.15.31:0/1920584786 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3e8014030 con 0x7fc3f01037c0 2024-09-16T00:40:49.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.758+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc3f0062630 con 0x7fc3dc0a32d0 2024-09-16T00:40:49.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:49 smithi031 ceph-mon[30956]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:49 smithi031 ceph-mon[30956]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.760+0000 7fc3e6ffd640 1 -- 172.21.15.31:0/1920584786 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc3f0062630 con 0x7fc3dc0a32d0 2024-09-16T00:40:49.761 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T00:40:49.761 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T00:40:49.761 INFO:teuthology.orchestra.run.smithi031.stdout:smithi081 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400N7400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T00:40:49.761 INFO:teuthology.orchestra.run.smithi031.stdout:smithi102 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.761+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3dc0a32d0 msgr2=0x7fc3dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.761+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3dc0a32d0 0x7fc3dc0a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec031400 comp rx=0 tx=0).stop 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.761+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f01037c0 msgr2=0x7fc3f0074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.761+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f01037c0 0x7fc3f0074b20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800cad0 tx=0x7fc3e800cf90 comp rx=0 tx=0).stop 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.762+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 shutdown_connections 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.762+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f01037c0 0x7fc3f0074b20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.762+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3dc0a32d0 0x7fc3dc0a5790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.762+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f0102e00 0x7fc3f00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.762+0000 7fc3f7a87640 1 --2- 172.21.15.31:0/1920584786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3f0101c00 0x7fc3f0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.762+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 >> 172.21.15.31:0/1920584786 conn(0x7fc3f00fd390 msgr2=0x7fc3f0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.762+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 shutdown_connections 2024-09-16T00:40:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:49.762+0000 7fc3f7a87640 1 -- 172.21.15.31:0/1920584786 wait complete. 2024-09-16T00:40:49.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:49 smithi102 ceph-mon[35364]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:49.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:49 smithi102 ceph-mon[35364]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:49.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:49 smithi081 ceph-mon[35539]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:49.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:49 smithi081 ceph-mon[35539]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:50.559 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:40:50.568 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:40:50.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T00:40:50.834 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 -- 172.21.15.31:0/803102600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103270 msgr2=0x7f18d8103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 --2- 172.21.15.31:0/803102600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103270 0x7f18d8103670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009900 tx=0x7f18c802a410 comp rx=0 tx=0).stop 2024-09-16T00:40:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 -- 172.21.15.31:0/803102600 shutdown_connections 2024-09-16T00:40:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 --2- 172.21.15.31:0/803102600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d8100390 0x7f18d810a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 --2- 172.21.15.31:0/803102600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d8103c40 0x7f18d80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 --2- 172.21.15.31:0/803102600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103270 0x7f18d8103670 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 -- 172.21.15.31:0/803102600 >> 172.21.15.31:0/803102600 conn(0x7f18d80fba80 msgr2=0x7f18d80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 -- 172.21.15.31:0/803102600 shutdown_connections 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 -- 172.21.15.31:0/803102600 wait complete. 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.332+0000 7f18de85a640 1 Processor -- start 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18de85a640 1 -- start start 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18de85a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d8100390 0x7f18d8102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18de85a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d8103270 0x7f18d8100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18de85a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103c40 0x7f18d8101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18de85a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f18d810d930 con 0x7f18d8103c40 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18de85a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f18d810d7b0 con 0x7f18d8100390 2024-09-16T00:40:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18de85a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f18d810dab0 con 0x7f18d8103270 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18dcdf7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103c40 0x7f18d8101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18dcdf7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103c40 0x7f18d8101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58852/0 (socket says 172.21.15.31:58852) 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18dcdf7640 1 -- 172.21.15.31:0/108753483 learned_addr learned my addr 172.21.15.31:0/108753483 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18d7fff640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d8100390 0x7f18d8102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18d77fe640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d8103270 0x7f18d8100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18dcdf7640 1 -- 172.21.15.31:0/108753483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d8103270 msgr2=0x7f18d8100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18dcdf7640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d8103270 0x7f18d8100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18dcdf7640 1 -- 172.21.15.31:0/108753483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d8100390 msgr2=0x7f18d8102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18dcdf7640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d8100390 0x7f18d8102970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18dcdf7640 1 -- 172.21.15.31:0/108753483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d8101dc0 con 0x7f18d8103c40 2024-09-16T00:40:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.333+0000 7f18d7fff640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d8100390 0x7f18d8102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.334+0000 7f18dcdf7640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103c40 0x7f18d8101500 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f18d000b970 tx=0x7f18d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.334+0000 7f18d57fa640 1 -- 172.21.15.31:0/108753483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0019070 con 0x7f18d8103c40 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.334+0000 7f18d57fa640 1 -- 172.21.15.31:0/108753483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d000cb40 con 0x7f18d8103c40 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.334+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d81a20b0 con 0x7f18d8103c40 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.334+0000 7f18d57fa640 1 -- 172.21.15.31:0/108753483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0007650 con 0x7f18d8103c40 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.334+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d81a2640 con 0x7f18d8103c40 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.335+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d8109140 con 0x7f18d8103c40 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.336+0000 7f18d57fa640 1 -- 172.21.15.31:0/108753483 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f18d001c070 con 0x7f18d8103c40 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.337+0000 7f18d57fa640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f18c00a3350 0x7f18c00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.337+0000 7f18d7fff640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f18c00a3350 0x7f18c00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.337+0000 7f18d57fa640 1 -- 172.21.15.31:0/108753483 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f18d00cf2f0 con 0x7f18d8103c40 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.337+0000 7f18d7fff640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f18c00a3350 0x7f18c00a5810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18c802aa00 tx=0x7f18c8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.341+0000 7f18d57fa640 1 -- 172.21.15.31:0/108753483 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18d009bc30 con 0x7f18d8103c40 2024-09-16T00:40:51.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.556+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f18d802d7c0 con 0x7f18d8103c40 2024-09-16T00:40:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.556+0000 7f18d57fa640 1 -- 172.21.15.31:0/108753483 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3046 (secure 0 0 0) 0x7f18d009b530 con 0x7f18d8103c40 2024-09-16T00:40:51.558 INFO:teuthology.orchestra.run.smithi031.stdout:e20 2024-09-16T00:40:51.558 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-16T00:39:36:131244+0000 2024-09-16T00:40:51.558 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:40:51.558 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:40:51.558 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T00:40:51.558 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:51.558 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 20 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T00:38:53.576781+0000 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T00:39:35.954166+0000 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 300 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T00:40:51.559 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24482} 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3,4] 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24482 members: 24482 2024-09-16T00:40:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b{0:24482} state up:active seq 4 addr [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.i{0:24472} state up:standby-replay seq 1 addr [v2:172.21.15.102:6838/3021718993,v1:172.21.15.102:6839/3021718993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.31:6834/3753760611,v1:172.21.15.31:6835/3753760611] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.31:6836/3116907098,v1:172.21.15.31:6837/3116907098] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.g{-1:14589} state up:standby seq 1 addr [v2:172.21.15.31:6838/3406901465,v1:172.21.15.31:6839/3406901465] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a{-1:14601} state up:standby seq 1 addr [v2:172.21.15.31:6840/3311366678,v1:172.21.15.31:6841/3311366678] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.k{-1:24407} state up:standby seq 1 addr [v2:172.21.15.81:6832/1467259017,v1:172.21.15.81:6833/1467259017] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.561 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.102:6832/316287545,v1:172.21.15.102:6833/316287545] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.562 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e{-1:24425} state up:standby seq 1 addr [v2:172.21.15.81:6834/1522041920,v1:172.21.15.81:6835/1522041920] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.562 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.h{-1:24446} state up:standby seq 1 addr [v2:172.21.15.81:6836/1338256650,v1:172.21.15.81:6837/1338256650] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.562 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f{-1:24448} state up:standby seq 1 addr [v2:172.21.15.102:6834/1825124874,v1:172.21.15.102:6835/1825124874] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.562 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c{-1:24460} state up:standby seq 1 addr [v2:172.21.15.102:6836/272579536,v1:172.21.15.102:6837/272579536] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.558+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f18c00a3350 msgr2=0x7f18c00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.558+0000 7f18de85a640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f18c00a3350 0x7f18c00a5810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18c802aa00 tx=0x7f18c8031400 comp rx=0 tx=0).stop 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.558+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103c40 msgr2=0x7f18d8101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.558+0000 7f18de85a640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103c40 0x7f18d8101500 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f18d000b970 tx=0x7f18d000be30 comp rx=0 tx=0).stop 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.559+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 shutdown_connections 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.559+0000 7f18de85a640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18d8103c40 0x7f18d8101500 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.559+0000 7f18de85a640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f18c00a3350 0x7f18c00a5810 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.559+0000 7f18de85a640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d8103270 0x7f18d8100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.559+0000 7f18de85a640 1 --2- 172.21.15.31:0/108753483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d8100390 0x7f18d8102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.559+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 >> 172.21.15.31:0/108753483 conn(0x7f18d80fba80 msgr2=0x7f18d8106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.559+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 shutdown_connections 2024-09-16T00:40:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:51.559+0000 7f18de85a640 1 -- 172.21.15.31:0/108753483 wait complete. 2024-09-16T00:40:51.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:51 smithi102 ceph-mon[35364]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:51.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:51 smithi102 ceph-mon[35364]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:51.873 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:51 smithi031 ceph-mon[30956]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:51.873 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:51 smithi031 ceph-mon[30956]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:51.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:51 smithi081 ceph-mon[35539]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:51.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:51 smithi081 ceph-mon[35539]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:52.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-16T00:40:52.485 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:52.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:52 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/108753483' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:40:52.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:52 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/108753483' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:40:52.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:52 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/108753483' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:40:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 -- 172.21.15.31:0/4183218104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 msgr2=0x7fc350109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 --2- 172.21.15.31:0/4183218104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 0x7fc350109d50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc348009ca0 tx=0x7fc34802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 -- 172.21.15.31:0/4183218104 shutdown_connections 2024-09-16T00:40:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 --2- 172.21.15.31:0/4183218104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 0x7fc350109d50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 --2- 172.21.15.31:0/4183218104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc350105030 0x7fc350107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 --2- 172.21.15.31:0/4183218104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350102700 0x7fc350104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 -- 172.21.15.31:0/4183218104 >> 172.21.15.31:0/4183218104 conn(0x7fc3500fc280 msgr2=0x7fc3500fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 -- 172.21.15.31:0/4183218104 shutdown_connections 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 -- 172.21.15.31:0/4183218104 wait complete. 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.984+0000 7fc355410640 1 Processor -- start 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc355410640 1 -- start start 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc355410640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc350102700 0x7fc350197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc355410640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350105030 0x7fc3501983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc355410640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 0x7fc35019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc355410640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc35010d8b0 con 0x7fc350107960 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc355410640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc35010d730 con 0x7fc350102700 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc355410640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc35010da30 con 0x7fc350105030 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 0x7fc35019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 0x7fc35019f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58874/0 (socket says 172.21.15.31:58874) 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34f7fe640 1 -- 172.21.15.31:0/4189603562 learned_addr learned my addr 172.21.15.31:0/4189603562 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc350102700 0x7fc350197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34e7fc640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350105030 0x7fc3501983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34f7fe640 1 -- 172.21.15.31:0/4189603562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350105030 msgr2=0x7fc3501983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34f7fe640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350105030 0x7fc3501983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34f7fe640 1 -- 172.21.15.31:0/4189603562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc350102700 msgr2=0x7fc350197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34f7fe640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc350102700 0x7fc350197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34f7fe640 1 -- 172.21.15.31:0/4189603562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35019fae0 con 0x7fc350107960 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.985+0000 7fc34effd640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc350102700 0x7fc350197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.986+0000 7fc34f7fe640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 0x7fc35019f470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc3480319f0 tx=0x7fc34802f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.986+0000 7fc337fff640 1 -- 172.21.15.31:0/4189603562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348003c70 con 0x7fc350107960 2024-09-16T00:40:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.986+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35019fd70 con 0x7fc350107960 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.986+0000 7fc337fff640 1 -- 172.21.15.31:0/4189603562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc348003e10 con 0x7fc350107960 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.986+0000 7fc337fff640 1 -- 172.21.15.31:0/4189603562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34802fd30 con 0x7fc350107960 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.986+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3501a0250 con 0x7fc350107960 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.987+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3500fdd70 con 0x7fc350107960 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.988+0000 7fc337fff640 1 -- 172.21.15.31:0/4189603562 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc348004040 con 0x7fc350107960 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.989+0000 7fc337fff640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3380a3300 0x7fc3380a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.989+0000 7fc34effd640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3380a3300 0x7fc3380a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.989+0000 7fc337fff640 1 -- 172.21.15.31:0/4189603562 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7fc3480f11a0 con 0x7fc350107960 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.989+0000 7fc34effd640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3380a3300 0x7fc3380a57c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc340005d00 tx=0x7fc34000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:52.993+0000 7fc337fff640 1 -- 172.21.15.31:0/4189603562 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3480bdb40 con 0x7fc350107960 2024-09-16T00:40:53.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.214+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc35002d7c0 con 0x7fc350107960 2024-09-16T00:40:53.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.215+0000 7fc337fff640 1 -- 172.21.15.31:0/4189603562 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5530 (secure 0 0 0) 0x7fc3480bd440 con 0x7fc350107960 2024-09-16T00:40:53.216 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 89 2024-09-16T00:40:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:fsid ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:40:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T00:31:04.619511+0000 2024-09-16T00:40:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T00:39:59.366751+0000 2024-09-16T00:40:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T00:40:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 27 2024-09-16T00:40:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-09-16T00:40:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-09-16T00:40:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-09-16T00:40:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client luminous 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 12 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6802/1435627868,v1:172.21.15.31:6803/1435627868] [v2:172.21.15.31:6804/1435627868,v1:172.21.15.31:6805/1435627868] exists,up d076c77e-9b90-46b1-8e52-eecb4d02d169 2024-09-16T00:40:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6800/3628874088,v1:172.21.15.81:6801/3628874088] [v2:172.21.15.81:6802/3628874088,v1:172.21.15.81:6803/3628874088] exists,up cc498a19-f3ac-4145-a121-81e710cd1558 2024-09-16T00:40:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 17 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6800/2755894572,v1:172.21.15.102:6801/2755894572] [v2:172.21.15.102:6802/2755894572,v1:172.21.15.102:6803/2755894572] exists,up 761075fe-1766-4924-afa3-6e2b1b3c3a37 2024-09-16T00:40:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 24 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6810/3958016219,v1:172.21.15.31:6811/3958016219] [v2:172.21.15.31:6812/3958016219,v1:172.21.15.31:6813/3958016219] exists,up 1f362961-16d5-4892-99df-3bec7a940e68 2024-09-16T00:40:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 32 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6808/59224774,v1:172.21.15.81:6809/59224774] [v2:172.21.15.81:6810/59224774,v1:172.21.15.81:6811/59224774] exists,up 2942a206-0e2f-4a98-99d3-efc5325578f3 2024-09-16T00:40:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 37 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6808/215828552,v1:172.21.15.102:6809/215828552] [v2:172.21.15.102:6810/215828552,v1:172.21.15.102:6811/215828552] exists,up 893456fd-1776-4a46-b16c-4590c38b3ee8 2024-09-16T00:40:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 44 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6818/1424825752,v1:172.21.15.31:6819/1424825752] [v2:172.21.15.31:6820/1424825752,v1:172.21.15.31:6821/1424825752] exists,up 749c866d-0f7f-4a78-ac75-5728ed7d49ca 2024-09-16T00:40:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 49 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6816/269260316,v1:172.21.15.81:6817/269260316] [v2:172.21.15.81:6818/269260316,v1:172.21.15.81:6819/269260316] exists,up 3f7eade8-dd2f-42ce-8b27-0c03f9172b79 2024-09-16T00:40:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 up in weight 1 up_from 56 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6816/304252744,v1:172.21.15.102:6817/304252744] [v2:172.21.15.102:6818/304252744,v1:172.21.15.102:6819/304252744] exists,up b2673057-c859-4b46-92af-c376f007503a 2024-09-16T00:40:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 up in weight 1 up_from 63 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6826/3465344894,v1:172.21.15.31:6827/3465344894] [v2:172.21.15.31:6828/3465344894,v1:172.21.15.31:6829/3465344894] exists,up cf12fcdc-621d-44c2-8863-e995d82fb5a7 2024-09-16T00:40:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 up in weight 1 up_from 68 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6824/1666424173,v1:172.21.15.81:6825/1666424173] [v2:172.21.15.81:6826/1666424173,v1:172.21.15.81:6827/1666424173] exists,up d229e783-ab63-44ce-816e-bf0a1949c5cc 2024-09-16T00:40:53.220 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 up in weight 1 up_from 75 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6824/1575052259,v1:172.21.15.102:6825/1575052259] [v2:172.21.15.102:6826/1575052259,v1:172.21.15.102:6827/1575052259] exists,up 1fddf945-ad39-4a5c-a40b-8ac262a7df75 2024-09-16T00:40:53.220 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.0 [8,4,7,11] 2024-09-16T00:40:53.220 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.a [2,11,8,1] 2024-09-16T00:40:53.220 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.c [2,4] 2024-09-16T00:40:53.220 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.e [2,1] 2024-09-16T00:40:53.220 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 4.c [9,10] 2024-09-16T00:40:53.220 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 4.f [5,8] 2024-09-16T00:40:53.220 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 4.14 [9,8] 2024-09-16T00:40:53.220 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 4.17 [9,8] 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2779988710 expires 2024-09-17T00:31:57.569184+0000 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/4219649043 expires 2024-09-17T00:31:57.569184+0000 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/945234876 expires 2024-09-17T00:31:33.279333+0000 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3876592548 expires 2024-09-17T00:31:33.279333+0000 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/4219649043 expires 2024-09-17T00:31:57.569184+0000 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/945234876 expires 2024-09-17T00:31:33.279333+0000 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/876405889 expires 2024-09-17T00:31:33.279333+0000 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2258150314 expires 2024-09-17T00:31:57.569184+0000 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/880818822 expires 2024-09-17T00:31:57.569184+0000 2024-09-16T00:40:53.221 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2222283837 expires 2024-09-17T00:31:33.279333+0000 2024-09-16T00:40:53.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3380a3300 msgr2=0x7fc3380a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3380a3300 0x7fc3380a57c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc340005d00 tx=0x7fc34000b410 comp rx=0 tx=0).stop 2024-09-16T00:40:53.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 msgr2=0x7fc35019f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:53.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 0x7fc35019f470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc3480319f0 tx=0x7fc34802f450 comp rx=0 tx=0).stop 2024-09-16T00:40:53.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 shutdown_connections 2024-09-16T00:40:53.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc350107960 0x7fc35019f470 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc3380a3300 0x7fc3380a57c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350105030 0x7fc3501983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 --2- 172.21.15.31:0/4189603562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc350102700 0x7fc350197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 >> 172.21.15.31:0/4189603562 conn(0x7fc3500fc280 msgr2=0x7fc350102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 shutdown_connections 2024-09-16T00:40:53.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:53.217+0000 7fc355410640 1 -- 172.21.15.31:0/4189603562 wait complete. 2024-09-16T00:40:53.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:53 smithi031 ceph-mon[30956]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:53.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:53 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:40:53.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:53 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4189603562' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:40:53.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:53 smithi102 ceph-mon[35364]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:53.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:53 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:40:53.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:53 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4189603562' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:40:53.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:53 smithi081 ceph-mon[35539]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:53.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:53 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:40:53.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:53 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4189603562' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:40:54.514 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-16T00:40:54.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:55.241 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:55 smithi031 ceph-mon[30956]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:55.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.325+0000 7ff97c5f6640 1 -- 172.21.15.31:0/1260150928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 msgr2=0x7ff97410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.325+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/1260150928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 0x7ff97410da40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009ca0 tx=0x7ff96c02a950 comp rx=0 tx=0).stop 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.325+0000 7ff97c5f6640 1 -- 172.21.15.31:0/1260150928 shutdown_connections 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.325+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/1260150928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 0x7ff97410da40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.325+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/1260150928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff974102570 0x7ff9741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.325+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/1260150928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974100e20 0x7ff974101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.325+0000 7ff97c5f6640 1 -- 172.21.15.31:0/1260150928 >> 172.21.15.31:0/1260150928 conn(0x7ff9740fc7d0 msgr2=0x7ff9740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.325+0000 7ff97c5f6640 1 -- 172.21.15.31:0/1260150928 shutdown_connections 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.325+0000 7ff97c5f6640 1 -- 172.21.15.31:0/1260150928 wait complete. 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97c5f6640 1 Processor -- start 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97c5f6640 1 -- start start 2024-09-16T00:40:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97c5f6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff974100e20 0x7ff9741984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97c5f6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974102570 0x7ff974198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97c5f6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 0x7ff97419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97c5f6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff974110f80 con 0x7ff974102ef0 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97c5f6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff974110e00 con 0x7ff974102570 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97c5f6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff974111100 con 0x7ff974100e20 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97ab93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 0x7ff97419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97ab93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 0x7ff97419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58892/0 (socket says 172.21.15.31:58892) 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff97ab93640 1 -- 172.21.15.31:0/976244949 learned_addr learned my addr 172.21.15.31:0/976244949 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.326+0000 7ff979b91640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974102570 0x7ff974198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff97a392640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff974100e20 0x7ff9741984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff97ab93640 1 -- 172.21.15.31:0/976244949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff974100e20 msgr2=0x7ff9741984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff97ab93640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff974100e20 0x7ff9741984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff97ab93640 1 -- 172.21.15.31:0/976244949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974102570 msgr2=0x7ff974198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff97ab93640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974102570 0x7ff974198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff97ab93640 1 -- 172.21.15.31:0/976244949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97419d4a0 con 0x7ff974102ef0 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff97ab93640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 0x7ff97419cdc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff96c02ae70 tx=0x7ff96c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff96b7fe640 1 -- 172.21.15.31:0/976244949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c034040 con 0x7ff974102ef0 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff96b7fe640 1 -- 172.21.15.31:0/976244949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff96c030920 con 0x7ff974102ef0 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97419d730 con 0x7ff974102ef0 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff96b7fe640 1 -- 172.21.15.31:0/976244949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c045480 con 0x7ff974102ef0 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.327+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9741a5070 con 0x7ff974102ef0 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.328+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9741021b0 con 0x7ff974102ef0 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.329+0000 7ff96b7fe640 1 -- 172.21.15.31:0/976244949 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff96c037020 con 0x7ff974102ef0 2024-09-16T00:40:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.330+0000 7ff96b7fe640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff9600a32d0 0x7ff9600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.331+0000 7ff97a392640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff9600a32d0 0x7ff9600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.331+0000 7ff96b7fe640 1 -- 172.21.15.31:0/976244949 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7ff96c0f0b80 con 0x7ff974102ef0 2024-09-16T00:40:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.331+0000 7ff97a392640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff9600a32d0 0x7ff9600a5790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff97000bbe0 tx=0x7ff97000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.335+0000 7ff96b7fe640 1 -- 172.21.15.31:0/976244949 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff96c0bd4c0 con 0x7ff974102ef0 2024-09-16T00:40:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:55 smithi081 ceph-mon[35539]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.553 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:55 smithi102 ceph-mon[35364]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:55.554 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.554 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.554 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.554 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.559+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff97410a0a0 con 0x7ff974102ef0 2024-09-16T00:40:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.560+0000 7ff96b7fe640 1 -- 172.21.15.31:0/976244949 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7ff96c0bcdc0 con 0x7ff974102ef0 2024-09-16T00:40:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:55.561 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:40:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:40:55.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.562+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff9600a32d0 msgr2=0x7ff9600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.562+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff9600a32d0 0x7ff9600a5790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff97000bbe0 tx=0x7ff97000b390 comp rx=0 tx=0).stop 2024-09-16T00:40:55.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.562+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 msgr2=0x7ff97419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.562+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 0x7ff97419cdc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff96c02ae70 tx=0x7ff96c02f5d0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.562+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 shutdown_connections 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.563+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff974102ef0 0x7ff97419cdc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.563+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974102570 0x7ff974198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.563+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff9600a32d0 0x7ff9600a5790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.563+0000 7ff97c5f6640 1 --2- 172.21.15.31:0/976244949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff974100e20 0x7ff9741984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.563+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 >> 172.21.15.31:0/976244949 conn(0x7ff9740fc7d0 msgr2=0x7ff97406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.563+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 shutdown_connections 2024-09-16T00:40:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:55.563+0000 7ff97c5f6640 1 -- 172.21.15.31:0/976244949 wait complete. 2024-09-16T00:40:56.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:40:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:56.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:56.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:56.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:56 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/976244949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:56.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:40:56.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:40:56.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:56 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:56.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:56.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:56.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:56 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/976244949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:56.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:40:56.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:40:56.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:56 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:56.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:56.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:56.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:56 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/976244949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:56.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:40:56.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:40:56.600 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:56 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:40:56.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.964+0000 7f348cc54640 1 -- 172.21.15.31:0/573000183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488100a80 msgr2=0x7f3488100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:56.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.964+0000 7f348cc54640 1 --2- 172.21.15.31:0/573000183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488100a80 0x7f3488100ee0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f347c009930 tx=0x7f347c02a420 comp rx=0 tx=0).stop 2024-09-16T00:40:56.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.964+0000 7f348cc54640 1 -- 172.21.15.31:0/573000183 shutdown_connections 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.964+0000 7f348cc54640 1 --2- 172.21.15.31:0/573000183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3488101420 0x7f348810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.964+0000 7f348cc54640 1 --2- 172.21.15.31:0/573000183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488100a80 0x7f3488100ee0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.964+0000 7f348cc54640 1 --2- 172.21.15.31:0/573000183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3488106a80 0x7f3488106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.964+0000 7f348cc54640 1 -- 172.21.15.31:0/573000183 >> 172.21.15.31:0/573000183 conn(0x7f34880fc7a0 msgr2=0x7f34880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.964+0000 7f348cc54640 1 -- 172.21.15.31:0/573000183 shutdown_connections 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.964+0000 7f348cc54640 1 -- 172.21.15.31:0/573000183 wait complete. 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.965+0000 7f348cc54640 1 Processor -- start 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.965+0000 7f348cc54640 1 -- start start 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.965+0000 7f348cc54640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3488100a80 0x7f3488198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.965+0000 7f348cc54640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3488101420 0x7f3488198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.965+0000 7f348cc54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488106a80 0x7f348819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.965+0000 7f348cc54640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3488110e80 con 0x7f3488106a80 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.965+0000 7f348cc54640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3488110d00 con 0x7f3488101420 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.965+0000 7f348cc54640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3488111000 con 0x7f3488100a80 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3486d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488106a80 0x7f348819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3486d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488106a80 0x7f348819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58902/0 (socket says 172.21.15.31:58902) 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3486d9d640 1 -- 172.21.15.31:0/2395521008 learned_addr learned my addr 172.21.15.31:0/2395521008 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f348659c640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3488100a80 0x7f3488198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3485d9b640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3488101420 0x7f3488198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3486d9d640 1 -- 172.21.15.31:0/2395521008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3488100a80 msgr2=0x7f3488198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3486d9d640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3488100a80 0x7f3488198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3486d9d640 1 -- 172.21.15.31:0/2395521008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3488101420 msgr2=0x7f3488198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:56.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3486d9d640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3488101420 0x7f3488198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3486d9d640 1 -- 172.21.15.31:0/2395521008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348819d3f0 con 0x7f3488106a80 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f3486d9d640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488106a80 0x7f348819cd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f348000cce0 tx=0x7f348000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f34777fe640 1 -- 172.21.15.31:0/2395521008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348000f450 con 0x7f3488106a80 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f34777fe640 1 -- 172.21.15.31:0/2395521008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f348000fad0 con 0x7f3488106a80 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348819d6e0 con 0x7f3488106a80 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f34777fe640 1 -- 172.21.15.31:0/2395521008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480018c10 con 0x7f3488106a80 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.966+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34881a4fc0 con 0x7f3488106a80 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.967+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3488100ee0 con 0x7f3488106a80 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.969+0000 7f34777fe640 1 -- 172.21.15.31:0/2395521008 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f348000f5f0 con 0x7f3488106a80 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.969+0000 7f34777fe640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34700a3350 0x7f34700a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.970+0000 7f348659c640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34700a3350 0x7f34700a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.970+0000 7f34777fe640 1 -- 172.21.15.31:0/2395521008 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9050+0+0 (secure 0 0 0) 0x7f34800d02d0 con 0x7f3488106a80 2024-09-16T00:40:56.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.970+0000 7f348659c640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34700a3350 0x7f34700a5810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3478009b50 tx=0x7f3478005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:56.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:56.974+0000 7f34777fe640 1 -- 172.21.15.31:0/2395521008 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3480014030 con 0x7f3488106a80 2024-09-16T00:40:57.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.186+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f348802d7c0 con 0x7f3488106a80 2024-09-16T00:40:57.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.186+0000 7f34777fe640 1 -- 172.21.15.31:0/2395521008 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f348009cc00 con 0x7f3488106a80 2024-09-16T00:40:57.188 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:57.188 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:40:57.189 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:40:57.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34700a3350 msgr2=0x7f34700a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34700a3350 0x7f34700a5810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3478009b50 tx=0x7f3478005f90 comp rx=0 tx=0).stop 2024-09-16T00:40:57.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488106a80 msgr2=0x7f348819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:57.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488106a80 0x7f348819cd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f348000cce0 tx=0x7f348000a620 comp rx=0 tx=0).stop 2024-09-16T00:40:57.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 shutdown_connections 2024-09-16T00:40:57.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3488106a80 0x7f348819cd10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3488101420 0x7f3488198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f34700a3350 0x7f34700a5810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 --2- 172.21.15.31:0/2395521008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3488100a80 0x7f3488198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:57.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 >> 172.21.15.31:0/2395521008 conn(0x7f34880fc7a0 msgr2=0x7f348806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:57.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 shutdown_connections 2024-09-16T00:40:57.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:57.189+0000 7f348cc54640 1 -- 172.21.15.31:0/2395521008 wait complete. 2024-09-16T00:40:57.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:57 smithi031 ceph-mon[30956]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:57.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:57 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2395521008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:57.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:40:57.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:57 smithi081 ceph-mon[35539]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:57.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:57 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2395521008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:58.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:57 smithi102 ceph-mon[35364]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:40:58.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:57 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2395521008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:40:58.199 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-16T00:40:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-16T00:40:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 6]}]: dispatch 2024-09-16T00:40:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-16T00:40:58.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.716+0000 7fc67edfe640 1 -- 172.21.15.31:0/3202808037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102570 msgr2=0x7fc6781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:58.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.716+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3202808037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102570 0x7fc6781029b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009900 tx=0x7fc66c02a480 comp rx=0 tx=0).stop 2024-09-16T00:40:58.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.716+0000 7fc67edfe640 1 -- 172.21.15.31:0/3202808037 shutdown_connections 2024-09-16T00:40:58.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.716+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3202808037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc678102ef0 0x7fc67810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.716+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3202808037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102570 0x7fc6781029b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.716+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3202808037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc678100e20 0x7fc678101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.716+0000 7fc67edfe640 1 -- 172.21.15.31:0/3202808037 >> 172.21.15.31:0/3202808037 conn(0x7fc6780fc7d0 msgr2=0x7fc6780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:58.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.716+0000 7fc67edfe640 1 -- 172.21.15.31:0/3202808037 shutdown_connections 2024-09-16T00:40:58.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.716+0000 7fc67edfe640 1 -- 172.21.15.31:0/3202808037 wait complete. 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67edfe640 1 Processor -- start 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67edfe640 1 -- start start 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67edfe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc678100e20 0x7fc6781984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67edfe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc678102570 0x7fc678198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67edfe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102ef0 0x7fc67819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67edfe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc678110fa0 con 0x7fc678102ef0 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67edfe640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc678110e20 con 0x7fc678102570 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67edfe640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc678111120 con 0x7fc678100e20 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67d39b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102ef0 0x7fc67819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67d39b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102ef0 0x7fc67819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56602/0 (socket says 172.21.15.31:56602) 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67d39b640 1 -- 172.21.15.31:0/3037762418 learned_addr learned my addr 172.21.15.31:0/3037762418 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:40:58.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67cb9a640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc678100e20 0x7fc6781984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc677fff640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc678102570 0x7fc678198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67d39b640 1 -- 172.21.15.31:0/3037762418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc678100e20 msgr2=0x7fc6781984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67d39b640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc678100e20 0x7fc6781984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67d39b640 1 -- 172.21.15.31:0/3037762418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc678102570 msgr2=0x7fc678198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67d39b640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc678102570 0x7fc678198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc67d39b640 1 -- 172.21.15.31:0/3037762418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67819d4a0 con 0x7fc678102ef0 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.717+0000 7fc677fff640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc678102570 0x7fc678198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.718+0000 7fc67d39b640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102ef0 0x7fc67819cdc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b970 tx=0x7fc67000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.718+0000 7fc675ffb640 1 -- 172.21.15.31:0/3037762418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670019070 con 0x7fc678102ef0 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.718+0000 7fc675ffb640 1 -- 172.21.15.31:0/3037762418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67000cb40 con 0x7fc678102ef0 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.718+0000 7fc675ffb640 1 -- 172.21.15.31:0/3037762418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670007650 con 0x7fc678102ef0 2024-09-16T00:40:58.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.718+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67819d790 con 0x7fc678102ef0 2024-09-16T00:40:58.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.718+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6781a5070 con 0x7fc678102ef0 2024-09-16T00:40:58.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.719+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc678101200 con 0x7fc678102ef0 2024-09-16T00:40:58.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.720+0000 7fc675ffb640 1 -- 172.21.15.31:0/3037762418 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fc67001c070 con 0x7fc678102ef0 2024-09-16T00:40:58.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.721+0000 7fc675ffb640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6600a32c0 0x7fc6600a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:58.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.721+0000 7fc67cb9a640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6600a32c0 0x7fc6600a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:58.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.721+0000 7fc675ffb640 1 -- 172.21.15.31:0/3037762418 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9174+0+0 (secure 0 0 0) 0x7fc6700cf2a0 con 0x7fc678102ef0 2024-09-16T00:40:58.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.721+0000 7fc67cb9a640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6600a32c0 0x7fc6600a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc668005e10 tx=0x7fc668005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:58.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.725+0000 7fc675ffb640 1 -- 172.21.15.31:0/3037762418 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc67009bc10 con 0x7fc678102ef0 2024-09-16T00:40:58.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.903+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc67810a0a0 con 0x7fc678102ef0 2024-09-16T00:40:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.905+0000 7fc675ffb640 1 -- 172.21.15.31:0/3037762418 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24766 (secure 0 0 0) 0x7fc67009b510 con 0x7fc678102ef0 2024-09-16T00:40:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:40:58.907 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:40:58.569182+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:40:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6600a32c0 msgr2=0x7fc6600a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6600a32c0 0x7fc6600a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc668005e10 tx=0x7fc668005da0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102ef0 msgr2=0x7fc67819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102ef0 0x7fc67819cdc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b970 tx=0x7fc67000be30 comp rx=0 tx=0).stop 2024-09-16T00:40:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 shutdown_connections 2024-09-16T00:40:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc678102ef0 0x7fc67819cdc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc678102570 0x7fc678198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc6600a32c0 0x7fc6600a5780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 --2- 172.21.15.31:0/3037762418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc678100e20 0x7fc6781984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 >> 172.21.15.31:0/3037762418 conn(0x7fc6780fc7d0 msgr2=0x7fc67806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 shutdown_connections 2024-09-16T00:40:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:40:58.908+0000 7fc67edfe640 1 -- 172.21.15.31:0/3037762418 wait complete. 2024-09-16T00:40:58.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-16T00:40:58.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-16T00:40:58.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 6]}]: dispatch 2024-09-16T00:40:58.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-16T00:40:59.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-16T00:40:59.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-16T00:40:59.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 6]}]: dispatch 2024-09-16T00:40:59.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-16T00:40:59.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:38:53.576781+0000', 'modified': '2024-09-16T00:39:35.954166+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6839/3021718993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3021718993}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3021718993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6839/2243721734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 2243721734}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 2243721734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482]} 2024-09-16T00:40:59.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T00:40:59.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:40:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:40:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:59 smithi031 ceph-mon[30956]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-16T00:40:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-16T00:40:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 6]}]': finished 2024-09-16T00:40:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-16T00:40:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:59 smithi031 ceph-mon[30956]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:40:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:59 smithi031 ceph-mon[30956]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:40:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:40:59 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3037762418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:40:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:59 smithi081 ceph-mon[35539]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:40:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-16T00:40:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-16T00:40:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 6]}]': finished 2024-09-16T00:40:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-16T00:40:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:59 smithi081 ceph-mon[35539]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:40:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:59 smithi081 ceph-mon[35539]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:40:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:40:59 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3037762418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:00.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:59 smithi102 ceph-mon[35364]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:00.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-16T00:41:00.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-16T00:41:00.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9, 4, 6]}]': finished 2024-09-16T00:41:00.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-16T00:41:00.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:59 smithi102 ceph-mon[35364]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:41:00.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:59 smithi102 ceph-mon[35364]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:41:00.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:40:59 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3037762418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 -- 172.21.15.31:0/777045285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3764103660 msgr2=0x7f376410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 --2- 172.21.15.31:0/777045285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3764103660 0x7f376410bb50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3754009900 tx=0x7f375402a480 comp rx=0 tx=0).stop 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 -- 172.21.15.31:0/777045285 shutdown_connections 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 --2- 172.21.15.31:0/777045285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376410c170 0x7f376410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 --2- 172.21.15.31:0/777045285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3764103660 0x7f376410bb50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 --2- 172.21.15.31:0/777045285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3764102d40 0x7f3764103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 -- 172.21.15.31:0/777045285 >> 172.21.15.31:0/777045285 conn(0x7f37640fc7a0 msgr2=0x7f37640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 -- 172.21.15.31:0/777045285 shutdown_connections 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 -- 172.21.15.31:0/777045285 wait complete. 2024-09-16T00:41:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 Processor -- start 2024-09-16T00:41:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.318+0000 7f376b4cb640 1 -- start start 2024-09-16T00:41:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f376b4cb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3764102d40 0x7f3764198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f376b4cb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3764103660 0x7f37641989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f376b4cb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376410c170 0x7f376419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f376b4cb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111f70 con 0x7f376410c170 2024-09-16T00:41:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f376b4cb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111df0 con 0x7f3764103660 2024-09-16T00:41:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f376b4cb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f37641120f0 con 0x7f3764102d40 2024-09-16T00:41:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769a68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376410c170 0x7f376419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769267640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3764102d40 0x7f3764198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3768a66640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3764103660 0x7f37641989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769267640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3764102d40 0x7f3764198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:36822/0 (socket says 172.21.15.31:36822) 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3768a66640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3764103660 0x7f37641989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:45458/0 (socket says 172.21.15.31:45458) 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769a68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376410c170 0x7f376419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56620/0 (socket says 172.21.15.31:56620) 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769267640 1 -- 172.21.15.31:0/2431184540 learned_addr learned my addr 172.21.15.31:0/2431184540 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769267640 1 -- 172.21.15.31:0/2431184540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3764103660 msgr2=0x7f37641989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769267640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3764103660 0x7f37641989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769267640 1 -- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376410c170 msgr2=0x7f376419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769267640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376410c170 0x7f376419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769267640 1 -- 172.21.15.31:0/2431184540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419d420 con 0x7f3764102d40 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769a68640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376410c170 0x7f376419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.319+0000 7f3769267640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3764102d40 0x7f3764198470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f376000d950 tx=0x7f376000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.320+0000 7f375a7fc640 1 -- 172.21.15.31:0/2431184540 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760017070 con 0x7f3764102d40 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.320+0000 7f376b4cb640 1 -- 172.21.15.31:0/2431184540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419d710 con 0x7f3764102d40 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.320+0000 7f375a7fc640 1 -- 172.21.15.31:0/2431184540 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3760009c30 con 0x7f3764102d40 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.320+0000 7f375a7fc640 1 -- 172.21.15.31:0/2431184540 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376000f6b0 con 0x7f3764102d40 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.320+0000 7f376b4cb640 1 -- 172.21.15.31:0/2431184540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641a4ff0 con 0x7f3764102d40 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.322+0000 7f375a7fc640 1 -- 172.21.15.31:0/2431184540 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f37600109d0 con 0x7f3764102d40 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.322+0000 7f376d8a4740 1 Processor -- start 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f376d8a4740 1 -- start start 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 0x55dd8a531db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5322f0 0x55dd8a53c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f376d8a4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a498090 con 0x55dd8a53cbb0 2024-09-16T00:41:00.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f376d8a4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a43fa70 con 0x55dd8a5319d0 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f376d8a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a494050 con 0x55dd8a5322f0 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3768a66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3768a66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56622/0 (socket says 172.21.15.31:56622) 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3768a66640 1 -- 172.21.15.31:0/407193271 learned_addr learned my addr 172.21.15.31:0/407193271 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3769267640 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5322f0 0x55dd8a53c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f375a7fc640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37500a3240 0x7f37500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3769a68640 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 0x55dd8a531db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3769a68640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37500a3240 0x7f37500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3768a66640 1 -- 172.21.15.31:0/407193271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5322f0 msgr2=0x55dd8a53c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3768a66640 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5322f0 0x55dd8a53c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3768a66640 1 -- 172.21.15.31:0/407193271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 msgr2=0x55dd8a531db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3768a66640 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 0x55dd8a531db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3768a66640 1 -- 172.21.15.31:0/407193271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd8a44c130 con 0x55dd8a53cbb0 2024-09-16T00:41:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f3769a68640 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 0x55dd8a531db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.323+0000 7f375a7fc640 1 -- 172.21.15.31:0/2431184540 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f37600cf880 con 0x7f3764102d40 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.324+0000 7f3768a66640 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53efa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3754009900 tx=0x7f3754005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985328ea95fc3b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.324+0000 7f3769a68640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37500a3240 0x7f37500a5700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f375c005c40 tx=0x7f375c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.324+0000 7f374f7fe640 1 -- 172.21.15.31:0/407193271 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754040070 con 0x55dd8a53cbb0 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.324+0000 7f374f7fe640 1 -- 172.21.15.31:0/407193271 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3754031870 con 0x55dd8a53cbb0 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.324+0000 7f374f7fe640 1 -- 172.21.15.31:0/407193271 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754035040 con 0x55dd8a53cbb0 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 -- 172.21.15.31:0/407193271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 msgr2=0x55dd8a53efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53efa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3754009900 tx=0x7f3754005be0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 -- 172.21.15.31:0/407193271 shutdown_connections 2024-09-16T00:41:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53efa0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5322f0 0x55dd8a53c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 --2- 172.21.15.31:0/407193271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 0x55dd8a531db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 -- 172.21.15.31:0/407193271 >> 172.21.15.31:0/407193271 conn(0x55dd8a52df60 msgr2=0x55dd8a52e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 -- 172.21.15.31:0/407193271 shutdown_connections 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 -- 172.21.15.31:0/407193271 wait complete. 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 -- start start 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.325+0000 7f376d8a4740 1 Processor -- start 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5319d0 0x55dd8a53c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 0x55dd8a539700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a539c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f376d8a4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a53f780 con 0x55dd8a53cbb0 2024-09-16T00:41:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f376d8a4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a452f20 con 0x55dd8a5322f0 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f376d8a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a53f900 con 0x55dd8a5319d0 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f376d8a4740 10 client.0 _ll_register_callbacks cb 0x55dd8a41c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3769a68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a539c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3769a68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a539c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56628/0 (socket says 172.21.15.31:56628) 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3769a68640 1 -- 172.21.15.31:0/468997191 learned_addr learned my addr 172.21.15.31:0/468997191 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3768a66640 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 0x55dd8a539700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3769267640 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5319d0 0x55dd8a53c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3769a68640 1 -- 172.21.15.31:0/468997191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5319d0 msgr2=0x55dd8a53c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3769a68640 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5319d0 0x55dd8a53c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3769a68640 1 -- 172.21.15.31:0/468997191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 msgr2=0x55dd8a539700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3769a68640 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 0x55dd8a539700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.326+0000 7f3769a68640 1 -- 172.21.15.31:0/468997191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd8a2092b0 con 0x55dd8a53cbb0 2024-09-16T00:41:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.327+0000 7f3769a68640 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a539c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f375c00fd20 tx=0x7f375c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccb74ceaf998647 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.327+0000 7f374cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.327+0000 7f374cff9640 1 -- 172.21.15.31:0/468997191 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c02c060 con 0x55dd8a53cbb0 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.327+0000 7f374cff9640 1 -- 172.21.15.31:0/468997191 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f375c00fe10 con 0x55dd8a53cbb0 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.327+0000 7f376d8a4740 1 -- 172.21.15.31:0/468997191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3754031870 con 0x55dd8a53cbb0 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.327+0000 7f374cff9640 1 -- 172.21.15.31:0/468997191 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f375c033050 con 0x55dd8a53cbb0 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.327+0000 7f376d8a4740 10 client.15123 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.327+0000 7f376d8a4740 1 -- 172.21.15.31:0/468997191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dd8a54b0c0 con 0x55dd8a53cbb0 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f374cff9640 1 -- 172.21.15.31:0/468997191 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3754031870 con 0x55dd8a53cbb0 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f374cff9640 1 -- 172.21.15.31:0/468997191 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f375c027770 con 0x55dd8a53cbb0 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f376d8a4740 10 client.15123 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f376d8a4740 10 client.15123 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f376d8a4740 20 client.15123 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f376d8a4740 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55dd8a54d750 0x55dd8a54ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f376d8a4740 4 client.15123 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f376d8a4740 1 -- 172.21.15.31:0/468997191 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd8a4a5600 con 0x55dd8a54d750 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f3769267640 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55dd8a54d750 0x55dd8a54ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f374cff9640 10 client.15123 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.328+0000 7f3769267640 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55dd8a54d750 0x55dd8a54ebc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954f01e996d36eb1 server_cookie=a4547ffa3deca204 in_seq=0 out_seq=0 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f374cff9640 1 -- 172.21.15.31:0/468997191 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd8a4a5600 con 0x55dd8a54d750 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f374cff9640 10 client.15123 handle_command_reply: tid=0 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f376d8a4740 1 client.15123 shutdown 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f376d8a4740 1 -- 172.21.15.31:0/468997191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 msgr2=0x55dd8a539c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f376d8a4740 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a539c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f375c00fd20 tx=0x7f375c004a40 comp rx=0 tx=0).stop 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f376d8a4740 1 -- 172.21.15.31:0/468997191 shutdown_connections 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f376d8a4740 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a53cbb0 0x55dd8a539c40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f376d8a4740 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 0x55dd8a539700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f376d8a4740 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55dd8a54d750 0x55dd8a54ebc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f374cff9640 0 client.15123 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:00.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f376d8a4740 1 --2- 172.21.15.31:0/468997191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a5319d0 0x55dd8a53c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.329+0000 7f376d8a4740 1 -- 172.21.15.31:0/468997191 >> 172.21.15.31:0/468997191 conn(0x55dd8a52df60 msgr2=0x55dd8a533c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.330+0000 7f376d8a4740 1 -- 172.21.15.31:0/468997191 shutdown_connections 2024-09-16T00:41:00.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.330+0000 7f376d8a4740 1 -- 172.21.15.31:0/468997191 wait complete. 2024-09-16T00:41:00.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.330+0000 7f376d8a4740 20 client.15123 trim_cache size 0 max 16384 2024-09-16T00:41:00.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f376d8a4740 1 Processor -- start 2024-09-16T00:41:00.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f376d8a4740 1 -- start start 2024-09-16T00:41:00.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5319d0 0x55dd8a54ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 0x55dd8a538fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f376d8a4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a54cf00 con 0x55dd8a5319d0 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f376d8a4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a54d5d0 con 0x55dd8a5322f0 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f376d8a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a54dca0 con 0x55dd8a53cbb0 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f3769267640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5319d0 0x55dd8a54ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f3769267640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5319d0 0x55dd8a54ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56640/0 (socket says 172.21.15.31:56640) 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f3769267640 1 -- 172.21.15.31:0/3991131043 learned_addr learned my addr 172.21.15.31:0/3991131043 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f3769a68640 1 --2- 172.21.15.31:0/3991131043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.348+0000 7f3768a66640 1 --2- 172.21.15.31:0/3991131043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 0x55dd8a538fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.349+0000 7f3769267640 1 -- 172.21.15.31:0/3991131043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 msgr2=0x55dd8a53c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.349+0000 7f3769267640 1 --2- 172.21.15.31:0/3991131043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.349+0000 7f3769267640 1 -- 172.21.15.31:0/3991131043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 msgr2=0x55dd8a538fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.349+0000 7f3769267640 1 --2- 172.21.15.31:0/3991131043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 0x55dd8a538fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.349+0000 7f3769267640 1 -- 172.21.15.31:0/3991131043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd8a549330 con 0x55dd8a5319d0 2024-09-16T00:41:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.349+0000 7f3769267640 1 --2- 172.21.15.31:0/3991131043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5319d0 0x55dd8a54ebc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f376009bad0 tx=0x7f37600cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d27e6c398d8f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.349+0000 7f374effd640 1 -- 172.21.15.31:0/3991131043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760020040 con 0x55dd8a5319d0 2024-09-16T00:41:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.349+0000 7f374effd640 1 -- 172.21.15.31:0/3991131043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f376009fe40 con 0x55dd8a5319d0 2024-09-16T00:41:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.349+0000 7f374effd640 1 -- 172.21.15.31:0/3991131043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760097890 con 0x55dd8a5319d0 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 -- 172.21.15.31:0/3991131043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5319d0 msgr2=0x55dd8a54ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 --2- 172.21.15.31:0/3991131043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5319d0 0x55dd8a54ebc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f376009bad0 tx=0x7f37600cf040 comp rx=0 tx=0).stop 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 -- 172.21.15.31:0/3991131043 shutdown_connections 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 --2- 172.21.15.31:0/3991131043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 0x55dd8a53c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 --2- 172.21.15.31:0/3991131043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5322f0 0x55dd8a538fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 --2- 172.21.15.31:0/3991131043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5319d0 0x55dd8a54ebc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 -- 172.21.15.31:0/3991131043 >> 172.21.15.31:0/3991131043 conn(0x55dd8a52df60 msgr2=0x55dd8a52e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 -- 172.21.15.31:0/3991131043 shutdown_connections 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 -- 172.21.15.31:0/3991131043 wait complete. 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 -- start start 2024-09-16T00:41:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.350+0000 7f376d8a4740 1 Processor -- start 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 0x55dd8a551ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5322f0 0x55dd8a5523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f376d8a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 0x55dd8a5b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f376d8a4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a54ea40 con 0x55dd8a5322f0 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f376d8a4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a54e370 con 0x55dd8a5319d0 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f376d8a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd8a54f110 con 0x55dd8a53cbb0 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f376d8a4740 10 client.0 _ll_register_callbacks cb 0x7f375c0115a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3768a66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5322f0 0x55dd8a5523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3768a66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5322f0 0x55dd8a5523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56652/0 (socket says 172.21.15.31:56652) 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3768a66640 1 -- 172.21.15.31:0/1291469163 learned_addr learned my addr 172.21.15.31:0/1291469163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3769267640 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 0x55dd8a551ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3769a68640 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 0x55dd8a5b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3768a66640 1 -- 172.21.15.31:0/1291469163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 msgr2=0x55dd8a5b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3768a66640 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 0x55dd8a5b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3768a66640 1 -- 172.21.15.31:0/1291469163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 msgr2=0x55dd8a551ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3768a66640 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 0x55dd8a551ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3768a66640 1 -- 172.21.15.31:0/1291469163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd8a552c90 con 0x55dd8a5322f0 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3769a68640 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 0x55dd8a5b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f3768a66640 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5322f0 0x55dd8a5523e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55dd8a52f2e0 tx=0x7f375402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17aaade7ae5456fe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.351+0000 7f374cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f374cff9640 1 -- 172.21.15.31:0/1291469163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754051060 con 0x55dd8a5322f0 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f376d8a4740 1 -- 172.21.15.31:0/1291469163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f376009fe40 con 0x55dd8a5322f0 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f374cff9640 1 -- 172.21.15.31:0/1291469163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f3754004040 con 0x55dd8a5322f0 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f374cff9640 1 -- 172.21.15.31:0/1291469163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3754058050 con 0x55dd8a5322f0 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f376d8a4740 10 client.15129 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f376d8a4740 1 -- 172.21.15.31:0/1291469163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dd8a5b7470 con 0x55dd8a5322f0 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f374cff9640 1 -- 172.21.15.31:0/1291469163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f376009fe40 con 0x55dd8a5322f0 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f374cff9640 1 -- 172.21.15.31:0/1291469163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f375403c440 con 0x55dd8a5322f0 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f376d8a4740 10 client.15129 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f376d8a4740 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f376d8a4740 20 client.15129 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f376d8a4740 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55dd8a54d750 0x55dd8a5ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f376d8a4740 4 client.15129 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.352+0000 7f376d8a4740 1 -- 172.21.15.31:0/1291469163 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "status"}) -- 0x55dd8a49fb10 con 0x55dd8a54d750 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f3769267640 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55dd8a54d750 0x55dd8a5ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f374cff9640 10 client.15129 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f3769267640 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55dd8a54d750 0x55dd8a5ba910 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8ae56e518e9182 server_cookie=c5c5503d231ed7d in_seq=0 out_seq=0 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f374cff9640 1 -- 172.21.15.31:0/1291469163 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dd8a49fb10 con 0x55dd8a54d750 2024-09-16T00:41:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f374cff9640 10 client.15129 handle_command_reply: tid=0 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f376d8a4740 1 client.15129 shutdown 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f376d8a4740 1 -- 172.21.15.31:0/1291469163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5322f0 msgr2=0x55dd8a5523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f376d8a4740 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5322f0 0x55dd8a5523e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55dd8a52f2e0 tx=0x7f375402a480 comp rx=0 tx=0).stop 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f376d8a4740 1 -- 172.21.15.31:0/1291469163 shutdown_connections 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f376d8a4740 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55dd8a54d750 0x55dd8a5ba910 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f376d8a4740 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd8a53cbb0 0x55dd8a5b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f376d8a4740 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd8a5322f0 0x55dd8a5523e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f376d8a4740 1 --2- 172.21.15.31:0/1291469163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd8a5319d0 0x55dd8a551ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f374cff9640 0 client.15129 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.353+0000 7f376d8a4740 1 -- 172.21.15.31:0/1291469163 >> 172.21.15.31:0/1291469163 conn(0x55dd8a52df60 msgr2=0x55dd8a537f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376d8a4740 1 -- 172.21.15.31:0/1291469163 shutdown_connections 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376d8a4740 1 -- 172.21.15.31:0/1291469163 wait complete. 2024-09-16T00:41:00.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376d8a4740 20 client.15129 trim_cache size 0 max 16384 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "ccb601ea-73c2-11ef-bceb-c7b262605968", 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24482, 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 126.348109802, 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 20, 2024-09-16T00:41:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 91, 2024-09-16T00:41:00.362 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-09-16T00:41:00.362 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 137.29584682699999 2024-09-16T00:41:00.362 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:41:00.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 -- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37500a3240 msgr2=0x7f37500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37500a3240 0x7f37500a5700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f375c005c40 tx=0x7f375c009500 comp rx=0 tx=0).stop 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 -- 172.21.15.31:0/2431184540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3764102d40 msgr2=0x7f3764198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3764102d40 0x7f3764198470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f376000d950 tx=0x7f376000de10 comp rx=0 tx=0).stop 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 -- 172.21.15.31:0/2431184540 shutdown_connections 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376410c170 0x7f376419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3764103660 0x7f37641989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37500a3240 0x7f37500a5700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 --2- 172.21.15.31:0/2431184540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3764102d40 0x7f3764198470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 -- 172.21.15.31:0/2431184540 >> 172.21.15.31:0/2431184540 conn(0x7f37640fc7a0 msgr2=0x7f37640fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 -- 172.21.15.31:0/2431184540 shutdown_connections 2024-09-16T00:41:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:00.354+0000 7f376b4cb640 1 -- 172.21.15.31:0/2431184540 wait complete. 2024-09-16T00:41:00.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:00 smithi031 ceph-mon[30956]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:41:00.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:00 smithi031 ceph-mon[30956]: pgmap v352: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T00:41:00.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:00 smithi081 ceph-mon[35539]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:41:00.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:00 smithi081 ceph-mon[35539]: pgmap v352: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T00:41:01.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:00 smithi102 ceph-mon[35364]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:41:01.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:00 smithi102 ceph-mon[35364]: pgmap v352: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 0 B/s wr, 1 op/s 2024-09-16T00:41:01.093 INFO:teuthology.run_tasks:Running task sequential... 2024-09-16T00:41:01.104 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-16T00:41:01.104 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-16T00:41:01.105 INFO:teuthology.task.print:Enabling mgr modules 2024-09-16T00:41:01.105 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T00:41:01.171 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T00:41:01.172 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-16T00:41:01.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:01.172 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T00:41:01.190 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-16T00:41:01.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:01.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:41:01.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:01.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-16T00:41:01.351 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-16T00:41:01.352 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:41:01.352 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:41:01.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:41:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:01.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:01 smithi031 ceph-mon[30956]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T00:41:01.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:01 smithi081 ceph-mon[35539]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T00:41:02.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:01 smithi102 ceph-mon[35364]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T00:41:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 -- 172.21.15.31:0/3533399509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c102790 msgr2=0x7f2d5c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 --2- 172.21.15.31:0/3533399509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c102790 0x7f2d5c1067e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009900 tx=0x7f2d4402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 -- 172.21.15.31:0/3533399509 shutdown_connections 2024-09-16T00:41:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 --2- 172.21.15.31:0/3533399509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d5c106f10 0x7f2d5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 --2- 172.21.15.31:0/3533399509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c102790 0x7f2d5c1067e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 --2- 172.21.15.31:0/3533399509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d5c101de0 0x7f2d5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 -- 172.21.15.31:0/3533399509 >> 172.21.15.31:0/3533399509 conn(0x7f2d5c0fd8c0 msgr2=0x7f2d5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 -- 172.21.15.31:0/3533399509 shutdown_connections 2024-09-16T00:41:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 -- 172.21.15.31:0/3533399509 wait complete. 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 Processor -- start 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.156+0000 7f2d63173640 1 -- start start 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d63173640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c101de0 0x7f2d5c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d63173640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d5c102790 0x7f2d5c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d63173640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d5c106f10 0x7f2d5c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d63173640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c111f70 con 0x7f2d5c101de0 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d63173640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c111df0 con 0x7f2d5c106f10 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d63173640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1120f0 con 0x7f2d5c102790 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d60f0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c101de0 0x7f2d5c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d60f0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c101de0 0x7f2d5c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56672/0 (socket says 172.21.15.31:56672) 2024-09-16T00:41:02.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d60f0f640 1 -- 172.21.15.31:0/2378633150 learned_addr learned my addr 172.21.15.31:0/2378633150 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d61710640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d5c106f10 0x7f2d5c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d5bfff640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d5c102790 0x7f2d5c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d60f0f640 1 -- 172.21.15.31:0/2378633150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d5c102790 msgr2=0x7f2d5c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d60f0f640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d5c102790 0x7f2d5c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d60f0f640 1 -- 172.21.15.31:0/2378633150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d5c106f10 msgr2=0x7f2d5c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d60f0f640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d5c106f10 0x7f2d5c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d60f0f640 1 -- 172.21.15.31:0/2378633150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c19d360 con 0x7f2d5c101de0 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d60f0f640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c101de0 0x7f2d5c1983b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00d950 tx=0x7f2d4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.157+0000 7f2d59ffb640 1 -- 172.21.15.31:0/2378633150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c017070 con 0x7f2d5c101de0 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.158+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5c19d650 con 0x7f2d5c101de0 2024-09-16T00:41:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.158+0000 7f2d59ffb640 1 -- 172.21.15.31:0/2378633150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4c009c30 con 0x7f2d5c101de0 2024-09-16T00:41:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.158+0000 7f2d59ffb640 1 -- 172.21.15.31:0/2378633150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00f6b0 con 0x7f2d5c101de0 2024-09-16T00:41:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.158+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5c1a4f30 con 0x7f2d5c101de0 2024-09-16T00:41:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.159+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5c1021c0 con 0x7f2d5c101de0 2024-09-16T00:41:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.160+0000 7f2d59ffb640 1 -- 172.21.15.31:0/2378633150 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2d4c0109d0 con 0x7f2d5c101de0 2024-09-16T00:41:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.161+0000 7f2d59ffb640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d480a3240 0x7f2d480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.161+0000 7f2d5bfff640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d480a3240 0x7f2d480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.161+0000 7f2d59ffb640 1 -- 172.21.15.31:0/2378633150 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f2d4c0cf820 con 0x7f2d5c101de0 2024-09-16T00:41:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.161+0000 7f2d5bfff640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d480a3240 0x7f2d480a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009230 tx=0x7f2d440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.165+0000 7f2d59ffb640 1 -- 172.21.15.31:0/2378633150 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d4c009de0 con 0x7f2d5c101de0 2024-09-16T00:41:02.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.378+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2d5c10a150 con 0x7f2d5c101de0 2024-09-16T00:41:02.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.379+0000 7f2d59ffb640 1 -- 172.21.15.31:0/2378633150 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+88 (secure 0 0 0) 0x7f2d4c09bb40 con 0x7f2d5c101de0 2024-09-16T00:41:02.380 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T00:41:02.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d480a3240 msgr2=0x7f2d480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d480a3240 0x7f2d480a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009230 tx=0x7f2d440058d0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c101de0 msgr2=0x7f2d5c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c101de0 0x7f2d5c1983b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00d950 tx=0x7f2d4c00de10 comp rx=0 tx=0).stop 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 shutdown_connections 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d5c106f10 0x7f2d5c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2d480a3240 0x7f2d480a5700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d5c102790 0x7f2d5c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 --2- 172.21.15.31:0/2378633150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d5c101de0 0x7f2d5c1983b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 >> 172.21.15.31:0/2378633150 conn(0x7f2d5c0fd8c0 msgr2=0x7f2d5c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 shutdown_connections 2024-09-16T00:41:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:02.381+0000 7f2d63173640 1 -- 172.21.15.31:0/2378633150 wait complete. 2024-09-16T00:41:02.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:02 smithi031 ceph-mon[30956]: pgmap v353: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-16T00:41:02.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:02 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2378633150' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:41:02.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:02 smithi081 ceph-mon[35539]: pgmap v353: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-16T00:41:02.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:02 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2378633150' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:41:03.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:02 smithi102 ceph-mon[35364]: pgmap v353: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-16T00:41:03.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:02 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2378633150' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:41:03.113 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T00:41:03.113 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:41:03.114 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-09-16T00:41:03.114 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T00:41:03.114 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:41:03.114 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T00:41:03.114 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:41:03.114 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:41:03.114 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:41:03.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:03.114 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-09-16T00:41:03.132 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:41:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-09-16T00:41:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-09-16T00:41:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T00:41:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42191sec preferred_lft 42191sec 2024-09-16T00:41:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-16T00:41:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:41:03.134 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:41:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-09-16T00:41:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-09-16T00:41:03.134 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:41:03.134 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:41:03.134 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:41:03.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:41:03.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-09-16T00:41:03.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-09-16T00:41:03.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:41:03.135 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:41:03.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:41:03.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:41:03.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:03.279 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:41:03.304 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-09-16T00:41:03.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:03.306 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-09-16T00:41:03.363 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-09-16T00:41:03.364 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:41:03.365 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T00:41:03.365 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T00:41:03.365 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:41:03.365 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:41:03.365 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:41:03.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T00:41:03.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T00:41:03.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T00:41:03.365 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:41:03.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:41:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:41:03.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:03.494 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T00:41:03.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:03.512 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-09-16T00:41:03.566 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:41:03.566 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:41:03.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:41:03.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T00:41:03.567 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:41:03.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:41:03.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:41:03.686 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T00:41:03.687 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:41:03.687 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:41:03.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T00:41:03.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:41:03.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T00:41:03.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T00:41:03.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T00:41:03.687 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:41:03.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:41:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:41:03.805 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:41:03.805 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:41:03.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-09-16T00:41:03.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:41:03.805 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:41:03.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:41:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:41:03.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:41:03.884 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T00:41:03.900 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:41:03.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:41:03.901 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:41:03.956 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-16T00:41:03.956 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-16T00:41:03.956 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/ff2f5246-15d0-407a-b34e-c330a0c83c3c 2024-09-16T00:41:03.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:03.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/ff2f5246-15d0-407a-b34e-c330a0c83c3c /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-16T00:41:04.042 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-16T00:41:04.042 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-16T00:41:04.042 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-09-16T00:41:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-16T00:41:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-16T00:41:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-09-16T00:41:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.81:3300,172.21.15.102:3300" 2024-09-16T00:41:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:3300,172.21.15.81:3300,172.21.15.102:3300:/volumes/qa/sv_1/ff2f5246-15d0-407a-b34e-c330a0c83c3c 2024-09-16T00:41:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=ccb601ea-73c2-11ef-bceb-c7b262605968" will pass to kernel 2024-09-16T00:41:04.187 INFO:teuthology.orchestra.run.smithi031.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T00:41:04.187 INFO:teuthology.orchestra.run.smithi031.stdout: You just mounted a file system that supports labels which does not 2024-09-16T00:41:04.187 INFO:teuthology.orchestra.run.smithi031.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T00:41:04.187 INFO:teuthology.orchestra.run.smithi031.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T00:41:04.188 INFO:teuthology.orchestra.run.smithi031.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T00:41:04.189 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T00:41:04.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:41:04.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:41:04.216 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:41:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:41:04.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:41:04.315 INFO:teuthology.orchestra.run.smithi031.stdout:client24860 2024-09-16T00:41:04.315 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:41:04.315 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:41:04.335 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:41:04.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:41:04.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:41:04.433 INFO:teuthology.orchestra.run.smithi031.stdout:client24860 2024-09-16T00:41:04.434 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:41:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:41:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:41:04.491 INFO:teuthology.orchestra.run.smithi031.stdout:client24860 2024-09-16T00:41:04.491 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968.client24860/status') 2024-09-16T00:41:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:41:04.551 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.24860 (3)172.21.15.31:0/3525521419 2024-09-16T00:41:04.551 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2024-09-16T00:41:04.551 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2024-09-16T00:41:04.551 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2024-09-16T00:41:04.551 INFO:teuthology.orchestra.run.smithi031.stderr:71 bytes copied, 2.3613e-05 s, 3.0 MB/s 2024-09-16T00:41:04.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:41:04.559 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-16T00:41:04.571 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-16T00:41:04.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:41:04.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.684+0000 7f6c21e5e640 1 -- 172.21.15.31:0/137945623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c1036e0 msgr2=0x7f6c1c109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:04.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.684+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/137945623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c1036e0 0x7f6c1c109f70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009ca0 tx=0x7f6c1402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.684+0000 7f6c21e5e640 1 -- 172.21.15.31:0/137945623 shutdown_connections 2024-09-16T00:41:04.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.684+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/137945623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c1036e0 0x7f6c1c109f70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.684+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/137945623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c102d20 0x7f6c1c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.684+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/137945623 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c1c101b30 0x7f6c1c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.684+0000 7f6c21e5e640 1 -- 172.21.15.31:0/137945623 >> 172.21.15.31:0/137945623 conn(0x7f6c1c0fd2b0 msgr2=0x7f6c1c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.684+0000 7f6c21e5e640 1 -- 172.21.15.31:0/137945623 shutdown_connections 2024-09-16T00:41:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.684+0000 7f6c21e5e640 1 -- 172.21.15.31:0/137945623 wait complete. 2024-09-16T00:41:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c21e5e640 1 Processor -- start 2024-09-16T00:41:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c21e5e640 1 -- start start 2024-09-16T00:41:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c21e5e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c101b30 0x7f6c1c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c21e5e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c102d20 0x7f6c1c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c21e5e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c1c1036e0 0x7f6c1c19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c21e5e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c10d7e0 con 0x7f6c1c102d20 2024-09-16T00:41:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c21e5e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c10d660 con 0x7f6c1c1036e0 2024-09-16T00:41:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c21e5e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c10d960 con 0x7f6c1c101b30 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c1affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c102d20 0x7f6c1c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c1affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c102d20 0x7f6c1c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56692/0 (socket says 172.21.15.31:56692) 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c1affd640 1 -- 172.21.15.31:0/765054726 learned_addr learned my addr 172.21.15.31:0/765054726 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.685+0000 7f6c1b7fe640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c101b30 0x7f6c1c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c1bfff640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c1c1036e0 0x7f6c1c19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c1affd640 1 -- 172.21.15.31:0/765054726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c101b30 msgr2=0x7f6c1c197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c1affd640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c101b30 0x7f6c1c197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c1affd640 1 -- 172.21.15.31:0/765054726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c1c1036e0 msgr2=0x7f6c1c19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c1affd640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c1c1036e0 0x7f6c1c19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c1affd640 1 -- 172.21.15.31:0/765054726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c19fa00 con 0x7f6c1c102d20 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c1affd640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c102d20 0x7f6c1c198320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00c890 tx=0x7f6c0c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c18ff9640 1 -- 172.21.15.31:0/765054726 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c004220 con 0x7f6c1c102d20 2024-09-16T00:41:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c19fcf0 con 0x7f6c1c102d20 2024-09-16T00:41:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c18ff9640 1 -- 172.21.15.31:0/765054726 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c0043c0 con 0x7f6c1c102d20 2024-09-16T00:41:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c18ff9640 1 -- 172.21.15.31:0/765054726 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0049e0 con 0x7f6c1c102d20 2024-09-16T00:41:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.686+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c1a01e0 con 0x7f6c1c102d20 2024-09-16T00:41:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.687+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1c108e50 con 0x7f6c1c102d20 2024-09-16T00:41:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.690+0000 7f6c18ff9640 1 -- 172.21.15.31:0/765054726 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6c0c0079b0 con 0x7f6c1c102d20 2024-09-16T00:41:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.690+0000 7f6c18ff9640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c080a3250 0x7f6c080a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.690+0000 7f6c1b7fe640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c080a3250 0x7f6c080a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.690+0000 7f6c18ff9640 1 -- 172.21.15.31:0/765054726 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f6c0c0cfcd0 con 0x7f6c1c102d20 2024-09-16T00:41:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.690+0000 7f6c1b7fe640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c080a3250 0x7f6c080a5710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c04005fd0 tx=0x7f6c0400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.694+0000 7f6c18ff9640 1 -- 172.21.15.31:0/765054726 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c0c014030 con 0x7f6c1c102d20 2024-09-16T00:41:04.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.870+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c1c02da10 con 0x7f6c1c102d20 2024-09-16T00:41:04.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.872+0000 7f6c18ff9640 1 -- 172.21.15.31:0/765054726 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24766 (secure 0 0 0) 0x7f6c0c09c5e0 con 0x7f6c1c102d20 2024-09-16T00:41:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:04.874 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:40:59.575845+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:41:04.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:04 smithi031 ceph-mon[30956]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2024-09-16T00:41:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c080a3250 msgr2=0x7f6c080a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c080a3250 0x7f6c080a5710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c04005fd0 tx=0x7f6c0400b410 comp rx=0 tx=0).stop 2024-09-16T00:41:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c102d20 msgr2=0x7f6c1c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c102d20 0x7f6c1c198320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00c890 tx=0x7f6c0c00cd50 comp rx=0 tx=0).stop 2024-09-16T00:41:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 shutdown_connections 2024-09-16T00:41:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c1c1036e0 0x7f6c1c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c102d20 0x7f6c1c198320 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c080a3250 0x7f6c080a5710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 --2- 172.21.15.31:0/765054726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c1c101b30 0x7f6c1c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 >> 172.21.15.31:0/765054726 conn(0x7f6c1c0fd2b0 msgr2=0x7f6c1c106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 shutdown_connections 2024-09-16T00:41:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.875+0000 7f6c21e5e640 1 -- 172.21.15.31:0/765054726 wait complete. 2024-09-16T00:41:04.887 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:34:12.067708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:38:49.806646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:38:51.945160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-16T00:38:58.457307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-16T00:41:04.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T00:41:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.993+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/367843210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 msgr2=0x7f9ff8103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.993+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/367843210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 0x7f9ff8103190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009900 tx=0x7f9fe002a410 comp rx=0 tx=0).stop 2024-09-16T00:41:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.993+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/367843210 shutdown_connections 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.993+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/367843210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff81036d0 0x7f9ff8109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.993+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/367843210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 0x7f9ff8103190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.993+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/367843210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ff8101b20 0x7f9ff8101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.993+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/367843210 >> 172.21.15.31:0/367843210 conn(0x7f9ff80fd2a0 msgr2=0x7f9ff80ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.993+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/367843210 shutdown_connections 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.993+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/367843210 wait complete. 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ffdc4c640 1 Processor -- start 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ffdc4c640 1 -- start start 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ffdc4c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff8101b20 0x7f9ff8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ffdc4c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 0x7f9ff80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ffdc4c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ff81036d0 0x7f9ff8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ffdc4c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810d7f0 con 0x7f9ff8102d10 2024-09-16T00:41:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ffdc4c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810d670 con 0x7f9ff81036d0 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ffdc4c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810d970 con 0x7f9ff8101b20 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9feffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 0x7f9ff80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9feffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 0x7f9ff80745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56710/0 (socket says 172.21.15.31:56710) 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9feffff640 1 -- 172.21.15.31:0/926163866 learned_addr learned my addr 172.21.15.31:0/926163866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ff7fff640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ff81036d0 0x7f9ff8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9ff77fe640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff8101b20 0x7f9ff8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9feffff640 1 -- 172.21.15.31:0/926163866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff8101b20 msgr2=0x7f9ff8075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9feffff640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff8101b20 0x7f9ff8075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9feffff640 1 -- 172.21.15.31:0/926163866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ff81036d0 msgr2=0x7f9ff8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9feffff640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ff81036d0 0x7f9ff8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9feffff640 1 -- 172.21.15.31:0/926163866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff80753b0 con 0x7f9ff8102d10 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.994+0000 7f9feffff640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 0x7f9ff80745e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0031440 tx=0x7f9fe0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.995+0000 7f9ff57fa640 1 -- 172.21.15.31:0/926163866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe003f030 con 0x7f9ff8102d10 2024-09-16T00:41:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.995+0000 7f9ff57fa640 1 -- 172.21.15.31:0/926163866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe0003ce0 con 0x7f9ff8102d10 2024-09-16T00:41:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.995+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff81a1f10 con 0x7f9ff8102d10 2024-09-16T00:41:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.995+0000 7f9ff57fa640 1 -- 172.21.15.31:0/926163866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0035070 con 0x7f9ff8102d10 2024-09-16T00:41:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.995+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff81a2370 con 0x7f9ff8102d10 2024-09-16T00:41:04.999 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:04 smithi081 ceph-mon[35539]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2024-09-16T00:41:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.996+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff8108f10 con 0x7f9ff8102d10 2024-09-16T00:41:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.997+0000 7f9ff57fa640 1 -- 172.21.15.31:0/926163866 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9fe0042030 con 0x7f9ff8102d10 2024-09-16T00:41:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.998+0000 7f9ff57fa640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fe40a3250 0x7f9fe40a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.998+0000 7f9ff77fe640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fe40a3250 0x7f9fe40a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.998+0000 7f9ff57fa640 1 -- 172.21.15.31:0/926163866 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f9fe00f0110 con 0x7f9ff8102d10 2024-09-16T00:41:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:04.998+0000 7f9ff77fe640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fe40a3250 0x7f9fe40a5710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8102b70 tx=0x7f9fe800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.003+0000 7f9ff57fa640 1 -- 172.21.15.31:0/926163866 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fe00bc430 con 0x7f9ff8102d10 2024-09-16T00:41:05.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:04 smithi102 ceph-mon[35364]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2024-09-16T00:41:05.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.179+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ff8062e90 con 0x7f9ff8102d10 2024-09-16T00:41:05.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.180+0000 7f9ff57fa640 1 -- 172.21.15.31:0/926163866 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f9fe00c0730 con 0x7f9ff8102d10 2024-09-16T00:41:05.181 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-16T00:41:05.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fe40a3250 msgr2=0x7f9fe40a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fe40a3250 0x7f9fe40a5710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8102b70 tx=0x7f9fe800b390 comp rx=0 tx=0).stop 2024-09-16T00:41:05.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 msgr2=0x7f9ff80745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 0x7f9ff80745e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0031440 tx=0x7f9fe0032ce0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 shutdown_connections 2024-09-16T00:41:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ff81036d0 0x7f9ff8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff8102d10 0x7f9ff80745e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fe40a3250 0x7f9fe40a5710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 --2- 172.21.15.31:0/926163866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff8101b20 0x7f9ff8075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 >> 172.21.15.31:0/926163866 conn(0x7f9ff80fd2a0 msgr2=0x7f9ff81060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 shutdown_connections 2024-09-16T00:41:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.182+0000 7f9ffdc4c640 1 -- 172.21.15.31:0/926163866 wait complete. 2024-09-16T00:41:05.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T00:41:05.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.300+0000 7f04576b7640 1 -- 172.21.15.31:0/2179728526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450104ee0 msgr2=0x7f04501072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.300+0000 7f04576b7640 1 --2- 172.21.15.31:0/2179728526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450104ee0 0x7f04501072d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0440009900 tx=0x7f044002a410 comp rx=0 tx=0).stop 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.300+0000 7f04576b7640 1 -- 172.21.15.31:0/2179728526 shutdown_connections 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.300+0000 7f04576b7640 1 --2- 172.21.15.31:0/2179728526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0450107810 0x7f0450109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.300+0000 7f04576b7640 1 --2- 172.21.15.31:0/2179728526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450104ee0 0x7f04501072d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.300+0000 7f04576b7640 1 --2- 172.21.15.31:0/2179728526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04501025b0 0x7f04501049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.300+0000 7f04576b7640 1 -- 172.21.15.31:0/2179728526 >> 172.21.15.31:0/2179728526 conn(0x7f04500fc1b0 msgr2=0x7f04500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.300+0000 7f04576b7640 1 -- 172.21.15.31:0/2179728526 shutdown_connections 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.300+0000 7f04576b7640 1 -- 172.21.15.31:0/2179728526 wait complete. 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f04576b7640 1 Processor -- start 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f04576b7640 1 -- start start 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f04576b7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04501025b0 0x7f0450197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f04576b7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0450104ee0 0x7f04501983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f04576b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450107810 0x7f045019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f04576b7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f045010d7c0 con 0x7f0450107810 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f04576b7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f045010d640 con 0x7f04501025b0 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f04576b7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f045010d940 con 0x7f0450104ee0 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455c54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450107810 0x7f045019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455c54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450107810 0x7f045019f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56718/0 (socket says 172.21.15.31:56718) 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455c54640 1 -- 172.21.15.31:0/1829344757 learned_addr learned my addr 172.21.15.31:0/1829344757 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455453640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04501025b0 0x7f0450197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0454c52640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0450104ee0 0x7f04501983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455c54640 1 -- 172.21.15.31:0/1829344757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0450104ee0 msgr2=0x7f04501983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455c54640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0450104ee0 0x7f04501983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455c54640 1 -- 172.21.15.31:0/1829344757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04501025b0 msgr2=0x7f0450197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455c54640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04501025b0 0x7f0450197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455c54640 1 -- 172.21.15.31:0/1829344757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045019fac0 con 0x7f0450107810 2024-09-16T00:41:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0454c52640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0450104ee0 0x7f04501983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.301+0000 7f0455c54640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450107810 0x7f045019f450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f044800e9d0 tx=0x7f044800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.302+0000 7f04467fc640 1 -- 172.21.15.31:0/1829344757 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448004d20 con 0x7f0450107810 2024-09-16T00:41:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.302+0000 7f04467fc640 1 -- 172.21.15.31:0/1829344757 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0448005020 con 0x7f0450107810 2024-09-16T00:41:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.302+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045019fdb0 con 0x7f0450107810 2024-09-16T00:41:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.302+0000 7f04467fc640 1 -- 172.21.15.31:0/1829344757 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448011690 con 0x7f0450107810 2024-09-16T00:41:05.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.302+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04501a0220 con 0x7f0450107810 2024-09-16T00:41:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.303+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04500fdca0 con 0x7f0450107810 2024-09-16T00:41:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.305+0000 7f04467fc640 1 -- 172.21.15.31:0/1829344757 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0448005540 con 0x7f0450107810 2024-09-16T00:41:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.305+0000 7f04467fc640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f043c0a3250 0x7f043c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.305+0000 7f04467fc640 1 -- 172.21.15.31:0/1829344757 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f04480d0270 con 0x7f0450107810 2024-09-16T00:41:05.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.305+0000 7f0455453640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f043c0a3250 0x7f043c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.306+0000 7f0455453640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f043c0a3250 0x7f043c0a5710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f044c00abe0 tx=0x7f044c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.309+0000 7f04467fc640 1 -- 172.21.15.31:0/1829344757 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0448016030 con 0x7f0450107810 2024-09-16T00:41:05.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.485+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0450062e90 con 0x7f0450107810 2024-09-16T00:41:05.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.485+0000 7f04467fc640 1 -- 172.21.15.31:0/1829344757 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f044809cbe0 con 0x7f0450107810 2024-09-16T00:41:05.485 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-09-16T00:41:05.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f043c0a3250 msgr2=0x7f043c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f043c0a3250 0x7f043c0a5710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f044c00abe0 tx=0x7f044c00a2d0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450107810 msgr2=0x7f045019f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450107810 0x7f045019f450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f044800e9d0 tx=0x7f044800edd0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 shutdown_connections 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0450107810 0x7f045019f450 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f043c0a3250 0x7f043c0a5710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0450104ee0 0x7f04501983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 --2- 172.21.15.31:0/1829344757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04501025b0 0x7f0450197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 >> 172.21.15.31:0/1829344757 conn(0x7f04500fc1b0 msgr2=0x7f0450103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 shutdown_connections 2024-09-16T00:41:05.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.487+0000 7f04576b7640 1 -- 172.21.15.31:0/1829344757 wait complete. 2024-09-16T00:41:05.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T00:41:05.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 -- 172.21.15.31:0/2443208613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5730102d10 msgr2=0x7f5730103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 --2- 172.21.15.31:0/2443208613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5730102d10 0x7f5730103190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5718009900 tx=0x7f571802a410 comp rx=0 tx=0).stop 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 -- 172.21.15.31:0/2443208613 shutdown_connections 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 --2- 172.21.15.31:0/2443208613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57301036d0 0x7f5730109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 --2- 172.21.15.31:0/2443208613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5730102d10 0x7f5730103190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 --2- 172.21.15.31:0/2443208613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5730101b20 0x7f5730101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 -- 172.21.15.31:0/2443208613 >> 172.21.15.31:0/2443208613 conn(0x7f57300fd2a0 msgr2=0x7f57300ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 -- 172.21.15.31:0/2443208613 shutdown_connections 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 -- 172.21.15.31:0/2443208613 wait complete. 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.605+0000 7f57358f1640 1 Processor -- start 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f57358f1640 1 -- start start 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f57358f1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5730101b20 0x7f5730197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f57358f1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5730102d10 0x7f57301983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f57358f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57301036d0 0x7f573019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f57358f1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f573010d7a0 con 0x7f57301036d0 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f57358f1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f573010d620 con 0x7f5730102d10 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f57358f1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f573010d920 con 0x7f5730101b20 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57301036d0 0x7f573019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57301036d0 0x7f573019f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56742/0 (socket says 172.21.15.31:56742) 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572f7fe640 1 -- 172.21.15.31:0/1226913167 learned_addr learned my addr 172.21.15.31:0/1226913167 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572effd640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5730101b20 0x7f5730197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572e7fc640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5730102d10 0x7f57301983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572f7fe640 1 -- 172.21.15.31:0/1226913167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5730101b20 msgr2=0x7f5730197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572f7fe640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5730101b20 0x7f5730197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572f7fe640 1 -- 172.21.15.31:0/1226913167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5730102d10 msgr2=0x7f57301983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572f7fe640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5730102d10 0x7f57301983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572f7fe640 1 -- 172.21.15.31:0/1226913167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573019fa80 con 0x7f57301036d0 2024-09-16T00:41:05.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.606+0000 7f572f7fe640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57301036d0 0x7f573019f410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f572800d750 tx=0x7f572800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.607+0000 7f57348ef640 1 -- 172.21.15.31:0/1226913167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728017070 con 0x7f57301036d0 2024-09-16T00:41:05.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.607+0000 7f57348ef640 1 -- 172.21.15.31:0/1226913167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5728005500 con 0x7f57301036d0 2024-09-16T00:41:05.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.607+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573019fd70 con 0x7f57301036d0 2024-09-16T00:41:05.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.607+0000 7f57348ef640 1 -- 172.21.15.31:0/1226913167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728004d90 con 0x7f57301036d0 2024-09-16T00:41:05.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.607+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57301a0280 con 0x7f57301036d0 2024-09-16T00:41:05.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.608+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5730108fb0 con 0x7f57301036d0 2024-09-16T00:41:05.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.609+0000 7f57348ef640 1 -- 172.21.15.31:0/1226913167 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5728004f30 con 0x7f57301036d0 2024-09-16T00:41:05.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.610+0000 7f57348ef640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f571c0a32d0 0x7f571c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.610+0000 7f572effd640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f571c0a32d0 0x7f571c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.610+0000 7f57348ef640 1 -- 172.21.15.31:0/1226913167 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f57280cf530 con 0x7f57301036d0 2024-09-16T00:41:05.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.610+0000 7f572effd640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f571c0a32d0 0x7f571c0a5790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5730102b70 tx=0x7f5720009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.614+0000 7f57348ef640 1 -- 172.21.15.31:0/1226913167 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f572809b850 con 0x7f57301036d0 2024-09-16T00:41:05.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.790+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5730062e90 con 0x7f57301036d0 2024-09-16T00:41:05.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.790+0000 7f57348ef640 1 -- 172.21.15.31:0/1226913167 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f572809fb50 con 0x7f57301036d0 2024-09-16T00:41:05.791 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-09-16T00:41:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f571c0a32d0 msgr2=0x7f571c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f571c0a32d0 0x7f571c0a5790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5730102b70 tx=0x7f5720009500 comp rx=0 tx=0).stop 2024-09-16T00:41:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57301036d0 msgr2=0x7f573019f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57301036d0 0x7f573019f410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f572800d750 tx=0x7f572800dc10 comp rx=0 tx=0).stop 2024-09-16T00:41:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 shutdown_connections 2024-09-16T00:41:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57301036d0 0x7f573019f410 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5730102d10 0x7f57301983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f571c0a32d0 0x7f571c0a5790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 --2- 172.21.15.31:0/1226913167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5730101b20 0x7f5730197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.792+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 >> 172.21.15.31:0/1226913167 conn(0x7f57300fd2a0 msgr2=0x7f5730106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.793+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 shutdown_connections 2024-09-16T00:41:05.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.793+0000 7f57358f1640 1 -- 172.21.15.31:0/1226913167 wait complete. 2024-09-16T00:41:05.803 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:05 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/765054726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:05.803 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:05 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/926163866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:41:05.803 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:05 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1829344757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:41:05.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-16T00:41:05.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.910+0000 7f4895465640 1 -- 172.21.15.31:0/2746136178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48901077e0 msgr2=0x7f4890109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.910+0000 7f4895465640 1 --2- 172.21.15.31:0/2746136178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48901077e0 0x7f4890109c50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4888009ca0 tx=0x7f488802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.910+0000 7f4895465640 1 -- 172.21.15.31:0/2746136178 shutdown_connections 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.910+0000 7f4895465640 1 --2- 172.21.15.31:0/2746136178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48901077e0 0x7f4890109c50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.910+0000 7f4895465640 1 --2- 172.21.15.31:0/2746136178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4890104eb0 0x7f48901072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.910+0000 7f4895465640 1 --2- 172.21.15.31:0/2746136178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4890102580 0x7f4890104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.910+0000 7f4895465640 1 -- 172.21.15.31:0/2746136178 >> 172.21.15.31:0/2746136178 conn(0x7f48900fc160 msgr2=0x7f48900fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.910+0000 7f4895465640 1 -- 172.21.15.31:0/2746136178 shutdown_connections 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.910+0000 7f4895465640 1 -- 172.21.15.31:0/2746136178 wait complete. 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f4895465640 1 Processor -- start 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f4895465640 1 -- start start 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f4895465640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4890102580 0x7f4890101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f4895465640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4890104eb0 0x7f4890100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f4895465640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48901077e0 0x7f4890100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f4895465640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f489010d7a0 con 0x7f4890102580 2024-09-16T00:41:05.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f4895465640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f489010d620 con 0x7f48901077e0 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f4895465640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f489010d920 con 0x7f4890104eb0 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4890102580 0x7f4890101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4890102580 0x7f4890101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56754/0 (socket says 172.21.15.31:56754) 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488effd640 1 -- 172.21.15.31:0/2223708976 learned_addr learned my addr 172.21.15.31:0/2223708976 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488f7fe640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48901077e0 0x7f4890100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488effd640 1 -- 172.21.15.31:0/2223708976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4890104eb0 msgr2=0x7f4890100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488effd640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4890104eb0 0x7f4890100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488effd640 1 -- 172.21.15.31:0/2223708976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48901077e0 msgr2=0x7f4890100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488effd640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48901077e0 0x7f4890100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488effd640 1 -- 172.21.15.31:0/2223708976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4890101300 con 0x7f4890102580 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.911+0000 7f488f7fe640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48901077e0 0x7f4890100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.912+0000 7f488effd640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4890102580 0x7f4890101ef0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f488000d950 tx=0x7f488000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.912+0000 7f4877fff640 1 -- 172.21.15.31:0/2223708976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880017070 con 0x7f4890102580 2024-09-16T00:41:05.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.912+0000 7f4877fff640 1 -- 172.21.15.31:0/2223708976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4880009c30 con 0x7f4890102580 2024-09-16T00:41:05.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.912+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48901a1f10 con 0x7f4890102580 2024-09-16T00:41:05.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.912+0000 7f4877fff640 1 -- 172.21.15.31:0/2223708976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488000f6b0 con 0x7f4890102580 2024-09-16T00:41:05.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.912+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48901a2370 con 0x7f4890102580 2024-09-16T00:41:05.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.913+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48900fdc50 con 0x7f4890102580 2024-09-16T00:41:05.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.914+0000 7f4877fff640 1 -- 172.21.15.31:0/2223708976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f48800109d0 con 0x7f4890102580 2024-09-16T00:41:05.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.915+0000 7f4877fff640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f48780a3250 0x7f48780a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:05.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.915+0000 7f488e7fc640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f48780a3250 0x7f48780a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:05.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.915+0000 7f4877fff640 1 -- 172.21.15.31:0/2223708976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f48800cf8c0 con 0x7f4890102580 2024-09-16T00:41:05.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.915+0000 7f488e7fc640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f48780a3250 0x7f48780a5710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4890101c60 tx=0x7f4884005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:05.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:05.918+0000 7f4877fff640 1 -- 172.21.15.31:0/2223708976 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4880009de0 con 0x7f4890102580 2024-09-16T00:41:05.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:05 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/765054726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:05.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:05 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/926163866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:41:05.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:05 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1829344757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:41:06.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:05 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/765054726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:06.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:05 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/926163866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:41:06.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:05 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1829344757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:41:06.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.094+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f489002da30 con 0x7f4890102580 2024-09-16T00:41:06.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.094+0000 7f4877fff640 1 -- 172.21.15.31:0/2223708976 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f488009c230 con 0x7f4890102580 2024-09-16T00:41:06.095 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-09-16T00:41:06.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f48780a3250 msgr2=0x7f48780a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:06.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f48780a3250 0x7f48780a5710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4890101c60 tx=0x7f4884005c70 comp rx=0 tx=0).stop 2024-09-16T00:41:06.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4890102580 msgr2=0x7f4890101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:06.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4890102580 0x7f4890101ef0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f488000d950 tx=0x7f488000de10 comp rx=0 tx=0).stop 2024-09-16T00:41:06.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 shutdown_connections 2024-09-16T00:41:06.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48901077e0 0x7f4890100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:06.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f48780a3250 0x7f48780a5710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:06.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4890104eb0 0x7f4890100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:06.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 --2- 172.21.15.31:0/2223708976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4890102580 0x7f4890101ef0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:06.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 >> 172.21.15.31:0/2223708976 conn(0x7f48900fc160 msgr2=0x7f4890102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:06.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 shutdown_connections 2024-09-16T00:41:06.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.096+0000 7f4895465640 1 -- 172.21.15.31:0/2223708976 wait complete. 2024-09-16T00:41:06.107 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-16T00:41:06.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.869+0000 7f41407f8640 1 -- 172.21.15.31:0/1606880877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138106f10 msgr2=0x7f413810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.869+0000 7f41407f8640 1 --2- 172.21.15.31:0/1606880877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138106f10 0x7f413810ea90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4130009900 tx=0x7f413002a410 comp rx=0 tx=0).stop 2024-09-16T00:41:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.869+0000 7f41407f8640 1 -- 172.21.15.31:0/1606880877 shutdown_connections 2024-09-16T00:41:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.869+0000 7f41407f8640 1 --2- 172.21.15.31:0/1606880877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138106f10 0x7f413810ea90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.869+0000 7f41407f8640 1 --2- 172.21.15.31:0/1606880877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4138102790 0x7f41381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.869+0000 7f41407f8640 1 --2- 172.21.15.31:0/1606880877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4138101de0 0x7f41381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.869+0000 7f41407f8640 1 -- 172.21.15.31:0/1606880877 >> 172.21.15.31:0/1606880877 conn(0x7f41380fd8c0 msgr2=0x7f41380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.869+0000 7f41407f8640 1 -- 172.21.15.31:0/1606880877 shutdown_connections 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.869+0000 7f41407f8640 1 -- 172.21.15.31:0/1606880877 wait complete. 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f41407f8640 1 Processor -- start 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f41407f8640 1 -- start start 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f41407f8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4138101de0 0x7f41381983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f41407f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138102790 0x7f4138198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f41407f8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4138106f10 0x7f413819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f41407f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4138111f70 con 0x7f4138102790 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f41407f8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4138111df0 con 0x7f4138101de0 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f41407f8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f41381120f0 con 0x7f4138106f10 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f413dd93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138102790 0x7f4138198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f413dd93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138102790 0x7f4138198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56764/0 (socket says 172.21.15.31:56764) 2024-09-16T00:41:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f413dd93640 1 -- 172.21.15.31:0/464174409 learned_addr learned my addr 172.21.15.31:0/464174409 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.870+0000 7f413e594640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4138101de0 0x7f41381983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f413ed95640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4138106f10 0x7f413819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f413dd93640 1 -- 172.21.15.31:0/464174409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4138106f10 msgr2=0x7f413819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f413dd93640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4138106f10 0x7f413819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f413dd93640 1 -- 172.21.15.31:0/464174409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4138101de0 msgr2=0x7f41381983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f413dd93640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4138101de0 0x7f41381983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f413dd93640 1 -- 172.21.15.31:0/464174409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413819d3a0 con 0x7f4138102790 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f413dd93640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138102790 0x7f4138198930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f412c00d950 tx=0x7f412c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f412b7fe640 1 -- 172.21.15.31:0/464174409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c017030 con 0x7f4138102790 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f412b7fe640 1 -- 172.21.15.31:0/464174409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c004020 con 0x7f4138102790 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413819d690 con 0x7f4138102790 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f412b7fe640 1 -- 172.21.15.31:0/464174409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0047a0 con 0x7f4138102790 2024-09-16T00:41:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.871+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41381a4f70 con 0x7f4138102790 2024-09-16T00:41:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.872+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41381021c0 con 0x7f4138102790 2024-09-16T00:41:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.873+0000 7f412b7fe640 1 -- 172.21.15.31:0/464174409 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f412c0041d0 con 0x7f4138102790 2024-09-16T00:41:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.874+0000 7f412b7fe640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41240a32c0 0x7f41240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.875+0000 7f413e594640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41240a32c0 0x7f41240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.875+0000 7f412b7fe640 1 -- 172.21.15.31:0/464174409 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f412c0cfd10 con 0x7f4138102790 2024-09-16T00:41:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.875+0000 7f413e594640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41240a32c0 0x7f41240a5780 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4134005c40 tx=0x7f413400a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:06.878+0000 7f412b7fe640 1 -- 172.21.15.31:0/464174409 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f412c09c700 con 0x7f4138102790 2024-09-16T00:41:06.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:06 smithi081 ceph-mon[35539]: pgmap v355: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T00:41:06.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:06 smithi081 ceph-mon[35539]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-16T00:41:06.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:06 smithi081 ceph-mon[35539]: Cluster is now healthy 2024-09-16T00:41:06.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:06 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1226913167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:41:06.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:06 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2223708976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:41:07.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:06 smithi102 ceph-mon[35364]: pgmap v355: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T00:41:07.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:06 smithi102 ceph-mon[35364]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-16T00:41:07.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:06 smithi102 ceph-mon[35364]: Cluster is now healthy 2024-09-16T00:41:07.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:06 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1226913167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:41:07.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:06 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2223708976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:41:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.089+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f413802d7c0 con 0x7f4138102790 2024-09-16T00:41:07.089 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:06 smithi031 ceph-mon[30956]: pgmap v355: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T00:41:07.089 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:06 smithi031 ceph-mon[30956]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-16T00:41:07.090 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:06 smithi031 ceph-mon[30956]: Cluster is now healthy 2024-09-16T00:41:07.090 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:06 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1226913167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:41:07.090 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:06 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2223708976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:41:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.089+0000 7f412b7fe640 1 -- 172.21.15.31:0/464174409 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f412c09c000 con 0x7f4138102790 2024-09-16T00:41:07.091 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:07.091 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:41:07.091 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.092+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41240a32c0 msgr2=0x7f41240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.092+0000 7f41407f8640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41240a32c0 0x7f41240a5780 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4134005c40 tx=0x7f413400a780 comp rx=0 tx=0).stop 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.092+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138102790 msgr2=0x7f4138198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.092+0000 7f41407f8640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138102790 0x7f4138198930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f412c00d950 tx=0x7f412c00de10 comp rx=0 tx=0).stop 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.092+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 shutdown_connections 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.093+0000 7f41407f8640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41240a32c0 0x7f41240a5780 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.093+0000 7f41407f8640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4138106f10 0x7f413819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.093+0000 7f41407f8640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4138102790 0x7f4138198930 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.093+0000 7f41407f8640 1 --2- 172.21.15.31:0/464174409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4138101de0 0x7f41381983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.093+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 >> 172.21.15.31:0/464174409 conn(0x7f41380fd8c0 msgr2=0x7f41380feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.093+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 shutdown_connections 2024-09-16T00:41:07.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.093+0000 7f41407f8640 1 -- 172.21.15.31:0/464174409 wait complete. 2024-09-16T00:41:07.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:07 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/464174409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:07.794 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-16T00:41:07.794 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-16T00:41:07.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:41:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 -- 172.21.15.31:0/1854709432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 msgr2=0x7f8878101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 --2- 172.21.15.31:0/1854709432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 0x7f8878101b30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8870009ca0 tx=0x7f887002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 -- 172.21.15.31:0/1854709432 shutdown_connections 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 --2- 172.21.15.31:0/1854709432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 0x7f8878101b30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 --2- 172.21.15.31:0/1854709432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8878100ce0 0x7f8878101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 --2- 172.21.15.31:0/1854709432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88781003a0 0x7f88781007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 -- 172.21.15.31:0/1854709432 >> 172.21.15.31:0/1854709432 conn(0x7f88780fc190 msgr2=0x7f88780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 -- 172.21.15.31:0/1854709432 shutdown_connections 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 -- 172.21.15.31:0/1854709432 wait complete. 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 Processor -- start 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.905+0000 7f887fc8d640 1 -- start start 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887fc8d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88781003a0 0x7f887819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887fc8d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8878100ce0 0x7f887819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887fc8d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 0x7f88781a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887fc8d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f887810fa70 con 0x7f8878101680 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887fc8d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f887810f8f0 con 0x7f8878100ce0 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887fc8d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f887810fbf0 con 0x7f88781003a0 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887e22a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 0x7f88781a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887e22a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 0x7f88781a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60300/0 (socket says 172.21.15.31:60300) 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887e22a640 1 -- 172.21.15.31:0/3029874240 learned_addr learned my addr 172.21.15.31:0/3029874240 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887da29640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88781003a0 0x7f887819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887d228640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8878100ce0 0x7f887819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887e22a640 1 -- 172.21.15.31:0/3029874240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88781003a0 msgr2=0x7f887819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887e22a640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88781003a0 0x7f887819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887e22a640 1 -- 172.21.15.31:0/3029874240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8878100ce0 msgr2=0x7f887819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887e22a640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8878100ce0 0x7f887819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887e22a640 1 -- 172.21.15.31:0/3029874240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88781a1d40 con 0x7f8878101680 2024-09-16T00:41:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887d228640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8878100ce0 0x7f887819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f887e22a640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 0x7f88781a16d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8870003f40 tx=0x7f8870003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f886effd640 1 -- 172.21.15.31:0/3029874240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700316d0 con 0x7f8878101680 2024-09-16T00:41:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.906+0000 7f886effd640 1 -- 172.21.15.31:0/3029874240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8870031870 con 0x7f8878101680 2024-09-16T00:41:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.907+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88781a1fd0 con 0x7f8878101680 2024-09-16T00:41:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.907+0000 7f886effd640 1 -- 172.21.15.31:0/3029874240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887002fbb0 con 0x7f8878101680 2024-09-16T00:41:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.907+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88781a2510 con 0x7f8878101680 2024-09-16T00:41:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.908+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f887810b250 con 0x7f8878101680 2024-09-16T00:41:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.909+0000 7f886effd640 1 -- 172.21.15.31:0/3029874240 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f887002fd50 con 0x7f8878101680 2024-09-16T00:41:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.909+0000 7f886effd640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f88640a3250 0x7f88640a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.909+0000 7f887da29640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f88640a3250 0x7f88640a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.909+0000 7f886effd640 1 -- 172.21.15.31:0/3029874240 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f88700f1310 con 0x7f8878101680 2024-09-16T00:41:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.910+0000 7f887da29640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f88640a3250 0x7f88640a5710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8874009990 tx=0x7f8874009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:07.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:07.913+0000 7f886effd640 1 -- 172.21.15.31:0/3029874240 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88700bdc80 con 0x7f8878101680 2024-09-16T00:41:08.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:07 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/464174409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.091+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8878062630 con 0x7f88640a3250 2024-09-16T00:41:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.099+0000 7f886effd640 1 -- 172.21.15.31:0/3029874240 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340406 (secure 0 0 0) 0x7f8878062630 con 0x7f88640a3250 2024-09-16T00:41:08.104 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:08.106 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f88640a3250 msgr2=0x7f88640a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f88640a3250 0x7f88640a5710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8874009990 tx=0x7f8874009110 comp rx=0 tx=0).stop 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 msgr2=0x7f88781a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 0x7f88781a16d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8870003f40 tx=0x7f8870003f70 comp rx=0 tx=0).stop 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 shutdown_connections 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101680 0x7f88781a16d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8878100ce0 0x7f887819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f88640a3250 0x7f88640a5710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 --2- 172.21.15.31:0/3029874240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88781003a0 0x7f887819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 >> 172.21.15.31:0/3029874240 conn(0x7f88780fc190 msgr2=0x7f88780fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 shutdown_connections 2024-09-16T00:41:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.103+0000 7f887fc8d640 1 -- 172.21.15.31:0/3029874240 wait complete. 2024-09-16T00:41:08.114 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-16T00:41:07.685864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655468,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":325,"num_read_kb":274,"num_write":274,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":715,"ondisk_log_size":715,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400920,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540904,"statfs":{"total":1151940427776,"available":1151529885696,"internally_reserved":0,"allocated":14483456,"data_stored":8169866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":557,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.109893"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586150+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:40:59.586150+0000","last_peered":"2024-09-16T00:40:59.586150+0000","last_clean":"2024-09-16T00:40:59.586150+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:40:59.586150+0000","last_undegraded":"2024-09-16T00:40:59.586150+0000","last_fullsized":"2024-09-16T00:40:59.586150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589958+0000","last_change":"2024-09-16T00:40:59.589958+0000","last_active":"2024-09-16T00:40:59.589958+0000","last_peered":"2024-09-16T00:40:59.589958+0000","last_clean":"2024-09-16T00:40:59.589958+0000","last_became_active":"2024-09-16T00:40:59.589549+0000","last_became_peered":"2024-09-16T00:40:59.589549+0000","last_unstale":"2024-09-16T00:40:59.589958+0000","last_undegraded":"2024-09-16T00:40:59.589958+0000","last_fullsized":"2024-09-16T00:40:59.589958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:33.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588133+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:40:59.588133+0000","last_peered":"2024-09-16T00:40:59.588133+0000","last_clean":"2024-09-16T00:40:59.588133+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:40:59.588133+0000","last_undegraded":"2024-09-16T00:40:59.588133+0000","last_fullsized":"2024-09-16T00:40:59.588133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747727+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:40:59.747727+0000","last_peered":"2024-09-16T00:40:59.747727+0000","last_clean":"2024-09-16T00:40:59.747727+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:40:59.747727+0000","last_undegraded":"2024-09-16T00:40:59.747727+0000","last_fullsized":"2024-09-16T00:40:59.747727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162291+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:41:00.162291+0000","last_peered":"2024-09-16T00:41:00.162291+0000","last_clean":"2024-09-16T00:41:00.162291+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:41:00.162291+0000","last_undegraded":"2024-09-16T00:41:00.162291+0000","last_fullsized":"2024-09-16T00:41:00.162291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589935+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:40:59.589935+0000","last_peered":"2024-09-16T00:40:59.589935+0000","last_clean":"2024-09-16T00:40:59.589935+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:40:59.589935+0000","last_undegraded":"2024-09-16T00:40:59.589935+0000","last_fullsized":"2024-09-16T00:40:59.589935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586994+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:40:59.586994+0000","last_peered":"2024-09-16T00:40:59.586994+0000","last_clean":"2024-09-16T00:40:59.586994+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:40:59.586994+0000","last_undegraded":"2024-09-16T00:40:59.586994+0000","last_fullsized":"2024-09-16T00:40:59.586994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587478+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:40:59.587478+0000","last_peered":"2024-09-16T00:40:59.587478+0000","last_clean":"2024-09-16T00:40:59.587478+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:40:59.587478+0000","last_undegraded":"2024-09-16T00:40:59.587478+0000","last_fullsized":"2024-09-16T00:40:59.587478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590172+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:40:59.590172+0000","last_peered":"2024-09-16T00:40:59.590172+0000","last_clean":"2024-09-16T00:40:59.590172+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:40:59.590172+0000","last_undegraded":"2024-09-16T00:40:59.590172+0000","last_fullsized":"2024-09-16T00:40:59.590172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587026+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:40:59.587026+0000","last_peered":"2024-09-16T00:40:59.587026+0000","last_clean":"2024-09-16T00:40:59.587026+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:40:59.587026+0000","last_undegraded":"2024-09-16T00:40:59.587026+0000","last_fullsized":"2024-09-16T00:40:59.587026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.596631+0000","last_change":"2024-09-16T00:40:59.596631+0000","last_active":"2024-09-16T00:40:59.596631+0000","last_peered":"2024-09-16T00:40:59.596631+0000","last_clean":"2024-09-16T00:40:59.596631+0000","last_became_active":"2024-09-16T00:40:59.596165+0000","last_became_peered":"2024-09-16T00:40:59.596165+0000","last_unstale":"2024-09-16T00:40:59.596631+0000","last_undegraded":"2024-09-16T00:40:59.596631+0000","last_fullsized":"2024-09-16T00:40:59.596631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:56.918291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,1],"acting":[9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590072+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:40:59.590072+0000","last_peered":"2024-09-16T00:40:59.590072+0000","last_clean":"2024-09-16T00:40:59.590072+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:40:59.590072+0000","last_undegraded":"2024-09-16T00:40:59.590072+0000","last_fullsized":"2024-09-16T00:40:59.590072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592530+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:40:59.592530+0000","last_peered":"2024-09-16T00:40:59.592530+0000","last_clean":"2024-09-16T00:40:59.592530+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:40:59.592530+0000","last_undegraded":"2024-09-16T00:40:59.592530+0000","last_fullsized":"2024-09-16T00:40:59.592530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586212+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:40:59.586212+0000","last_peered":"2024-09-16T00:40:59.586212+0000","last_clean":"2024-09-16T00:40:59.586212+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:40:59.586212+0000","last_undegraded":"2024-09-16T00:40:59.586212+0000","last_fullsized":"2024-09-16T00:40:59.586212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590233+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:40:59.590233+0000","last_peered":"2024-09-16T00:40:59.590233+0000","last_clean":"2024-09-16T00:40:59.590233+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:40:59.590233+0000","last_undegraded":"2024-09-16T00:40:59.590233+0000","last_fullsized":"2024-09-16T00:40:59.590233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747322+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:40:59.747322+0000","last_peered":"2024-09-16T00:40:59.747322+0000","last_clean":"2024-09-16T00:40:59.747322+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:40:59.747322+0000","last_undegraded":"2024-09-16T00:40:59.747322+0000","last_fullsized":"2024-09-16T00:40:59.747322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747467+0000","last_change":"2024-09-16T00:39:59.591063+0000","last_active":"2024-09-16T00:40:59.747467+0000","last_peered":"2024-09-16T00:40:59.747467+0000","last_clean":"2024-09-16T00:40:59.747467+0000","last_became_active":"2024-09-16T00:39:59.590574+0000","last_became_peered":"2024-09-16T00:39:59.590574+0000","last_unstale":"2024-09-16T00:40:59.747467+0000","last_undegraded":"2024-09-16T00:40:59.747467+0000","last_fullsized":"2024-09-16T00:40:59.747467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:57.020592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590255+0000","last_change":"2024-09-16T00:39:59.595772+0000","last_active":"2024-09-16T00:40:59.590255+0000","last_peered":"2024-09-16T00:40:59.590255+0000","last_clean":"2024-09-16T00:40:59.590255+0000","last_became_active":"2024-09-16T00:39:59.590772+0000","last_became_peered":"2024-09-16T00:39:59.590772+0000","last_unstale":"2024-09-16T00:40:59.590255+0000","last_undegraded":"2024-09-16T00:40:59.590255+0000","last_fullsized":"2024-09-16T00:40:59.590255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:53.236019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747573+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:40:59.747573+0000","last_peered":"2024-09-16T00:40:59.747573+0000","last_clean":"2024-09-16T00:40:59.747573+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:40:59.747573+0000","last_undegraded":"2024-09-16T00:40:59.747573+0000","last_fullsized":"2024-09-16T00:40:59.747573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747545+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:40:59.747545+0000","last_peered":"2024-09-16T00:40:59.747545+0000","last_clean":"2024-09-16T00:40:59.747545+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:40:59.747545+0000","last_undegraded":"2024-09-16T00:40:59.747545+0000","last_fullsized":"2024-09-16T00:40:59.747545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585840+0000","last_change":"2024-09-16T00:39:59.386827+0000","last_active":"2024-09-16T00:40:59.585840+0000","last_peered":"2024-09-16T00:40:59.585840+0000","last_clean":"2024-09-16T00:40:59.585840+0000","last_became_active":"2024-09-16T00:39:59.386389+0000","last_became_peered":"2024-09-16T00:39:59.386389+0000","last_unstale":"2024-09-16T00:40:59.585840+0000","last_undegraded":"2024-09-16T00:40:59.585840+0000","last_fullsized":"2024-09-16T00:40:59.585840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:12:21.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585940+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:40:59.585940+0000","last_peered":"2024-09-16T00:40:59.585940+0000","last_clean":"2024-09-16T00:40:59.585940+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:40:59.585940+0000","last_undegraded":"2024-09-16T00:40:59.585940+0000","last_fullsized":"2024-09-16T00:40:59.585940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587363+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:40:59.587363+0000","last_peered":"2024-09-16T00:40:59.587363+0000","last_clean":"2024-09-16T00:40:59.587363+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:40:59.587363+0000","last_undegraded":"2024-09-16T00:40:59.587363+0000","last_fullsized":"2024-09-16T00:40:59.587363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590175+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:40:59.590175+0000","last_peered":"2024-09-16T00:40:59.590175+0000","last_clean":"2024-09-16T00:40:59.590175+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:40:59.590175+0000","last_undegraded":"2024-09-16T00:40:59.590175+0000","last_fullsized":"2024-09-16T00:40:59.590175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162114+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:41:00.162114+0000","last_peered":"2024-09-16T00:41:00.162114+0000","last_clean":"2024-09-16T00:41:00.162114+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:41:00.162114+0000","last_undegraded":"2024-09-16T00:41:00.162114+0000","last_fullsized":"2024-09-16T00:41:00.162114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590740+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:40:59.590740+0000","last_peered":"2024-09-16T00:40:59.590740+0000","last_clean":"2024-09-16T00:40:59.590740+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:40:59.590740+0000","last_undegraded":"2024-09-16T00:40:59.590740+0000","last_fullsized":"2024-09-16T00:40:59.590740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588415+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:40:59.588415+0000","last_peered":"2024-09-16T00:40:59.588415+0000","last_clean":"2024-09-16T00:40:59.588415+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:40:59.588415+0000","last_undegraded":"2024-09-16T00:40:59.588415+0000","last_fullsized":"2024-09-16T00:40:59.588415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747653+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:40:59.747653+0000","last_peered":"2024-09-16T00:40:59.747653+0000","last_clean":"2024-09-16T00:40:59.747653+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:40:59.747653+0000","last_undegraded":"2024-09-16T00:40:59.747653+0000","last_fullsized":"2024-09-16T00:40:59.747653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586160+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:40:59.586160+0000","last_peered":"2024-09-16T00:40:59.586160+0000","last_clean":"2024-09-16T00:40:59.586160+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:40:59.586160+0000","last_undegraded":"2024-09-16T00:40:59.586160+0000","last_fullsized":"2024-09-16T00:40:59.586160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747093+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:40:59.747093+0000","last_peered":"2024-09-16T00:40:59.747093+0000","last_clean":"2024-09-16T00:40:59.747093+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:40:59.747093+0000","last_undegraded":"2024-09-16T00:40:59.747093+0000","last_fullsized":"2024-09-16T00:40:59.747093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595527+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:40:59.595527+0000","last_peered":"2024-09-16T00:40:59.595527+0000","last_clean":"2024-09-16T00:40:59.595527+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:40:59.595527+0000","last_undegraded":"2024-09-16T00:40:59.595527+0000","last_fullsized":"2024-09-16T00:40:59.595527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587025+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:40:59.587025+0000","last_peered":"2024-09-16T00:40:59.587025+0000","last_clean":"2024-09-16T00:40:59.587025+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:40:59.587025+0000","last_undegraded":"2024-09-16T00:40:59.587025+0000","last_fullsized":"2024-09-16T00:40:59.587025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747410+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:40:59.747410+0000","last_peered":"2024-09-16T00:40:59.747410+0000","last_clean":"2024-09-16T00:40:59.747410+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:40:59.747410+0000","last_undegraded":"2024-09-16T00:40:59.747410+0000","last_fullsized":"2024-09-16T00:40:59.747410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747586+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:40:59.747586+0000","last_peered":"2024-09-16T00:40:59.747586+0000","last_clean":"2024-09-16T00:40:59.747586+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:40:59.747586+0000","last_undegraded":"2024-09-16T00:40:59.747586+0000","last_fullsized":"2024-09-16T00:40:59.747586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585296+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:40:59.585296+0000","last_peered":"2024-09-16T00:40:59.585296+0000","last_clean":"2024-09-16T00:40:59.585296+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:40:59.585296+0000","last_undegraded":"2024-09-16T00:40:59.585296+0000","last_fullsized":"2024-09-16T00:40:59.585296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586000+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:40:59.586000+0000","last_peered":"2024-09-16T00:40:59.586000+0000","last_clean":"2024-09-16T00:40:59.586000+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:40:59.586000+0000","last_undegraded":"2024-09-16T00:40:59.586000+0000","last_fullsized":"2024-09-16T00:40:59.586000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747487+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:40:59.747487+0000","last_peered":"2024-09-16T00:40:59.747487+0000","last_clean":"2024-09-16T00:40:59.747487+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:40:59.747487+0000","last_undegraded":"2024-09-16T00:40:59.747487+0000","last_fullsized":"2024-09-16T00:40:59.747487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592602+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:40:59.592602+0000","last_peered":"2024-09-16T00:40:59.592602+0000","last_clean":"2024-09-16T00:40:59.592602+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:40:59.592602+0000","last_undegraded":"2024-09-16T00:40:59.592602+0000","last_fullsized":"2024-09-16T00:40:59.592602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595212+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:40:59.595212+0000","last_peered":"2024-09-16T00:40:59.595212+0000","last_clean":"2024-09-16T00:40:59.595212+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:40:59.595212+0000","last_undegraded":"2024-09-16T00:40:59.595212+0000","last_fullsized":"2024-09-16T00:40:59.595212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.591482+0000","last_change":"2024-09-16T00:40:59.591482+0000","last_active":"2024-09-16T00:40:59.591482+0000","last_peered":"2024-09-16T00:40:59.591482+0000","last_clean":"2024-09-16T00:40:59.591482+0000","last_became_active":"2024-09-16T00:40:59.590978+0000","last_became_peered":"2024-09-16T00:40:59.590978+0000","last_unstale":"2024-09-16T00:40:59.591482+0000","last_undegraded":"2024-09-16T00:40:59.591482+0000","last_fullsized":"2024-09-16T00:40:59.591482+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:56.975117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594993+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:40:59.594993+0000","last_peered":"2024-09-16T00:40:59.594993+0000","last_clean":"2024-09-16T00:40:59.594993+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:40:59.594993+0000","last_undegraded":"2024-09-16T00:40:59.594993+0000","last_fullsized":"2024-09-16T00:40:59.594993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595325+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:40:59.595325+0000","last_peered":"2024-09-16T00:40:59.595325+0000","last_clean":"2024-09-16T00:40:59.595325+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:40:59.595325+0000","last_undegraded":"2024-09-16T00:40:59.595325+0000","last_fullsized":"2024-09-16T00:40:59.595325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594855+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:40:59.594855+0000","last_peered":"2024-09-16T00:40:59.594855+0000","last_clean":"2024-09-16T00:40:59.594855+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:40:59.594855+0000","last_undegraded":"2024-09-16T00:40:59.594855+0000","last_fullsized":"2024-09-16T00:40:59.594855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747209+0000","last_change":"2024-09-16T00:39:59.376963+0000","last_active":"2024-09-16T00:40:59.747209+0000","last_peered":"2024-09-16T00:40:59.747209+0000","last_clean":"2024-09-16T00:40:59.747209+0000","last_became_active":"2024-09-16T00:39:59.376538+0000","last_became_peered":"2024-09-16T00:39:59.376538+0000","last_unstale":"2024-09-16T00:40:59.747209+0000","last_undegraded":"2024-09-16T00:40:59.747209+0000","last_fullsized":"2024-09-16T00:40:59.747209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:02.077749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587857+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:40:59.587857+0000","last_peered":"2024-09-16T00:40:59.587857+0000","last_clean":"2024-09-16T00:40:59.587857+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:40:59.587857+0000","last_undegraded":"2024-09-16T00:40:59.587857+0000","last_fullsized":"2024-09-16T00:40:59.587857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747729+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:40:59.747729+0000","last_peered":"2024-09-16T00:40:59.747729+0000","last_clean":"2024-09-16T00:40:59.747729+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:40:59.747729+0000","last_undegraded":"2024-09-16T00:40:59.747729+0000","last_fullsized":"2024-09-16T00:40:59.747729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590591+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:40:59.590591+0000","last_peered":"2024-09-16T00:40:59.590591+0000","last_clean":"2024-09-16T00:40:59.590591+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:40:59.590591+0000","last_undegraded":"2024-09-16T00:40:59.590591+0000","last_fullsized":"2024-09-16T00:40:59.590591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592357+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:40:59.592357+0000","last_peered":"2024-09-16T00:40:59.592357+0000","last_clean":"2024-09-16T00:40:59.592357+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:40:59.592357+0000","last_undegraded":"2024-09-16T00:40:59.592357+0000","last_fullsized":"2024-09-16T00:40:59.592357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592684+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:40:59.592684+0000","last_peered":"2024-09-16T00:40:59.592684+0000","last_clean":"2024-09-16T00:40:59.592684+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:40:59.592684+0000","last_undegraded":"2024-09-16T00:40:59.592684+0000","last_fullsized":"2024-09-16T00:40:59.592684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592846+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:40:59.592846+0000","last_peered":"2024-09-16T00:40:59.592846+0000","last_clean":"2024-09-16T00:40:59.592846+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:40:59.592846+0000","last_undegraded":"2024-09-16T00:40:59.592846+0000","last_fullsized":"2024-09-16T00:40:59.592846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589988+0000","last_change":"2024-09-16T00:39:59.593442+0000","last_active":"2024-09-16T00:40:59.589988+0000","last_peered":"2024-09-16T00:40:59.589988+0000","last_clean":"2024-09-16T00:40:59.589988+0000","last_became_active":"2024-09-16T00:39:59.593039+0000","last_became_peered":"2024-09-16T00:39:59.593039+0000","last_unstale":"2024-09-16T00:40:59.589988+0000","last_undegraded":"2024-09-16T00:40:59.589988+0000","last_fullsized":"2024-09-16T00:40:59.589988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:36.644148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587311+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:40:59.587311+0000","last_peered":"2024-09-16T00:40:59.587311+0000","last_clean":"2024-09-16T00:40:59.587311+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:40:59.587311+0000","last_undegraded":"2024-09-16T00:40:59.587311+0000","last_fullsized":"2024-09-16T00:40:59.587311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592642+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:40:59.592642+0000","last_peered":"2024-09-16T00:40:59.592642+0000","last_clean":"2024-09-16T00:40:59.592642+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:40:59.592642+0000","last_undegraded":"2024-09-16T00:40:59.592642+0000","last_fullsized":"2024-09-16T00:40:59.592642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589435+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:40:59.589435+0000","last_peered":"2024-09-16T00:40:59.589435+0000","last_clean":"2024-09-16T00:40:59.589435+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:40:59.589435+0000","last_undegraded":"2024-09-16T00:40:59.589435+0000","last_fullsized":"2024-09-16T00:40:59.589435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161277+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:41:00.161277+0000","last_peered":"2024-09-16T00:41:00.161277+0000","last_clean":"2024-09-16T00:41:00.161277+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:41:00.161277+0000","last_undegraded":"2024-09-16T00:41:00.161277+0000","last_fullsized":"2024-09-16T00:41:00.161277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592960+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:40:59.592960+0000","last_peered":"2024-09-16T00:40:59.592960+0000","last_clean":"2024-09-16T00:40:59.592960+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:40:59.592960+0000","last_undegraded":"2024-09-16T00:40:59.592960+0000","last_fullsized":"2024-09-16T00:40:59.592960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595479+0000","last_change":"2024-09-16T00:39:59.594994+0000","last_active":"2024-09-16T00:40:59.595479+0000","last_peered":"2024-09-16T00:40:59.595479+0000","last_clean":"2024-09-16T00:40:59.595479+0000","last_became_active":"2024-09-16T00:39:59.590640+0000","last_became_peered":"2024-09-16T00:39:59.590640+0000","last_unstale":"2024-09-16T00:40:59.595479+0000","last_undegraded":"2024-09-16T00:40:59.595479+0000","last_fullsized":"2024-09-16T00:40:59.595479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:21.391494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587189+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:40:59.587189+0000","last_peered":"2024-09-16T00:40:59.587189+0000","last_clean":"2024-09-16T00:40:59.587189+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:40:59.587189+0000","last_undegraded":"2024-09-16T00:40:59.587189+0000","last_fullsized":"2024-09-16T00:40:59.587189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747514+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:40:59.747514+0000","last_peered":"2024-09-16T00:40:59.747514+0000","last_clean":"2024-09-16T00:40:59.747514+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:40:59.747514+0000","last_undegraded":"2024-09-16T00:40:59.747514+0000","last_fullsized":"2024-09-16T00:40:59.747514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586731+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:40:59.586731+0000","last_peered":"2024-09-16T00:40:59.586731+0000","last_clean":"2024-09-16T00:40:59.586731+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:40:59.586731+0000","last_undegraded":"2024-09-16T00:40:59.586731+0000","last_fullsized":"2024-09-16T00:40:59.586731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587479+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:40:59.587479+0000","last_peered":"2024-09-16T00:40:59.587479+0000","last_clean":"2024-09-16T00:40:59.587479+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:40:59.587479+0000","last_undegraded":"2024-09-16T00:40:59.587479+0000","last_fullsized":"2024-09-16T00:40:59.587479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588245+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:40:59.588245+0000","last_peered":"2024-09-16T00:40:59.588245+0000","last_clean":"2024-09-16T00:40:59.588245+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:40:59.588245+0000","last_undegraded":"2024-09-16T00:40:59.588245+0000","last_fullsized":"2024-09-16T00:40:59.588245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585828+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:40:59.585828+0000","last_peered":"2024-09-16T00:40:59.585828+0000","last_clean":"2024-09-16T00:40:59.585828+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:40:59.585828+0000","last_undegraded":"2024-09-16T00:40:59.585828+0000","last_fullsized":"2024-09-16T00:40:59.585828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595364+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:40:59.595364+0000","last_peered":"2024-09-16T00:40:59.595364+0000","last_clean":"2024-09-16T00:40:59.595364+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:40:59.595364+0000","last_undegraded":"2024-09-16T00:40:59.595364+0000","last_fullsized":"2024-09-16T00:40:59.595364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592178+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:40:59.592178+0000","last_peered":"2024-09-16T00:40:59.592178+0000","last_clean":"2024-09-16T00:40:59.592178+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:40:59.592178+0000","last_undegraded":"2024-09-16T00:40:59.592178+0000","last_fullsized":"2024-09-16T00:40:59.592178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590248+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:40:59.590248+0000","last_peered":"2024-09-16T00:40:59.590248+0000","last_clean":"2024-09-16T00:40:59.590248+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:40:59.590248+0000","last_undegraded":"2024-09-16T00:40:59.590248+0000","last_fullsized":"2024-09-16T00:40:59.590248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586021+0000","last_change":"2024-09-16T00:39:59.593568+0000","last_active":"2024-09-16T00:40:59.586021+0000","last_peered":"2024-09-16T00:40:59.586021+0000","last_clean":"2024-09-16T00:40:59.586021+0000","last_became_active":"2024-09-16T00:39:59.593209+0000","last_became_peered":"2024-09-16T00:39:59.593209+0000","last_unstale":"2024-09-16T00:40:59.586021+0000","last_undegraded":"2024-09-16T00:40:59.586021+0000","last_fullsized":"2024-09-16T00:40:59.586021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:56.516773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.596544+0000","last_change":"2024-09-16T00:40:59.596544+0000","last_active":"2024-09-16T00:40:59.596544+0000","last_peered":"2024-09-16T00:40:59.596544+0000","last_clean":"2024-09-16T00:40:59.596544+0000","last_became_active":"2024-09-16T00:40:59.596128+0000","last_became_peered":"2024-09-16T00:40:59.596128+0000","last_unstale":"2024-09-16T00:40:59.596544+0000","last_undegraded":"2024-09-16T00:40:59.596544+0000","last_fullsized":"2024-09-16T00:40:59.596544+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:22.691164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587645+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:40:59.587645+0000","last_peered":"2024-09-16T00:40:59.587645+0000","last_clean":"2024-09-16T00:40:59.587645+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:40:59.587645+0000","last_undegraded":"2024-09-16T00:40:59.587645+0000","last_fullsized":"2024-09-16T00:40:59.587645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747362+0000","last_change":"2024-09-16T00:39:59.380258+0000","last_active":"2024-09-16T00:40:59.747362+0000","last_peered":"2024-09-16T00:40:59.747362+0000","last_clean":"2024-09-16T00:40:59.747362+0000","last_became_active":"2024-09-16T00:39:59.379813+0000","last_became_peered":"2024-09-16T00:39:59.379813+0000","last_unstale":"2024-09-16T00:40:59.747362+0000","last_undegraded":"2024-09-16T00:40:59.747362+0000","last_fullsized":"2024-09-16T00:40:59.747362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:03.110305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595055+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:40:59.595055+0000","last_peered":"2024-09-16T00:40:59.595055+0000","last_clean":"2024-09-16T00:40:59.595055+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:40:59.595055+0000","last_undegraded":"2024-09-16T00:40:59.595055+0000","last_fullsized":"2024-09-16T00:40:59.595055+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588623+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:40:59.588623+0000","last_peered":"2024-09-16T00:40:59.588623+0000","last_clean":"2024-09-16T00:40:59.588623+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:40:59.588623+0000","last_undegraded":"2024-09-16T00:40:59.588623+0000","last_fullsized":"2024-09-16T00:40:59.588623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588982+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:40:59.588982+0000","last_peered":"2024-09-16T00:40:59.588982+0000","last_clean":"2024-09-16T00:40:59.588982+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:40:59.588982+0000","last_undegraded":"2024-09-16T00:40:59.588982+0000","last_fullsized":"2024-09-16T00:40:59.588982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586881+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:40:59.586881+0000","last_peered":"2024-09-16T00:40:59.586881+0000","last_clean":"2024-09-16T00:40:59.586881+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:40:59.586881+0000","last_undegraded":"2024-09-16T00:40:59.586881+0000","last_fullsized":"2024-09-16T00:40:59.586881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":265,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:05.289803+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:41:05.289803+0000","last_peered":"2024-09-16T00:41:05.289803+0000","last_clean":"2024-09-16T00:41:05.289803+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:41:05.289803+0000","last_undegraded":"2024-09-16T00:41:05.289803+0000","last_fullsized":"2024-09-16T00:41:05.289803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64068,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":193,"num_read_kb":159,"num_write":34,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585067+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:40:59.585067+0000","last_peered":"2024-09-16T00:40:59.585067+0000","last_clean":"2024-09-16T00:40:59.585067+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:40:59.585067+0000","last_undegraded":"2024-09-16T00:40:59.585067+0000","last_fullsized":"2024-09-16T00:40:59.585067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587571+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:40:59.587571+0000","last_peered":"2024-09-16T00:40:59.587571+0000","last_clean":"2024-09-16T00:40:59.587571+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:40:59.587571+0000","last_undegraded":"2024-09-16T00:40:59.587571+0000","last_fullsized":"2024-09-16T00:40:59.587571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589244+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:40:59.589244+0000","last_peered":"2024-09-16T00:40:59.589244+0000","last_clean":"2024-09-16T00:40:59.589244+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:40:59.589244+0000","last_undegraded":"2024-09-16T00:40:59.589244+0000","last_fullsized":"2024-09-16T00:40:59.589244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590297+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:40:59.590297+0000","last_peered":"2024-09-16T00:40:59.590297+0000","last_clean":"2024-09-16T00:40:59.590297+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:40:59.590297+0000","last_undegraded":"2024-09-16T00:40:59.590297+0000","last_fullsized":"2024-09-16T00:40:59.590297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162296+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:41:00.162296+0000","last_peered":"2024-09-16T00:41:00.162296+0000","last_clean":"2024-09-16T00:41:00.162296+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:41:00.162296+0000","last_undegraded":"2024-09-16T00:41:00.162296+0000","last_fullsized":"2024-09-16T00:41:00.162296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592831+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:40:59.592831+0000","last_peered":"2024-09-16T00:40:59.592831+0000","last_clean":"2024-09-16T00:40:59.592831+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:40:59.592831+0000","last_undegraded":"2024-09-16T00:40:59.592831+0000","last_fullsized":"2024-09-16T00:40:59.592831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584869+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:40:59.584869+0000","last_peered":"2024-09-16T00:40:59.584869+0000","last_clean":"2024-09-16T00:40:59.584869+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:40:59.584869+0000","last_undegraded":"2024-09-16T00:40:59.584869+0000","last_fullsized":"2024-09-16T00:40:59.584869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587926+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:40:59.587926+0000","last_peered":"2024-09-16T00:40:59.587926+0000","last_clean":"2024-09-16T00:40:59.587926+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:40:59.587926+0000","last_undegraded":"2024-09-16T00:40:59.587926+0000","last_fullsized":"2024-09-16T00:40:59.587926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747739+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:40:59.747739+0000","last_peered":"2024-09-16T00:40:59.747739+0000","last_clean":"2024-09-16T00:40:59.747739+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:40:59.747739+0000","last_undegraded":"2024-09-16T00:40:59.747739+0000","last_fullsized":"2024-09-16T00:40:59.747739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747139+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:40:59.747139+0000","last_peered":"2024-09-16T00:40:59.747139+0000","last_clean":"2024-09-16T00:40:59.747139+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:40:59.747139+0000","last_undegraded":"2024-09-16T00:40:59.747139+0000","last_fullsized":"2024-09-16T00:40:59.747139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161443+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:41:00.161443+0000","last_peered":"2024-09-16T00:41:00.161443+0000","last_clean":"2024-09-16T00:41:00.161443+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:41:00.161443+0000","last_undegraded":"2024-09-16T00:41:00.161443+0000","last_fullsized":"2024-09-16T00:41:00.161443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584969+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:40:59.584969+0000","last_peered":"2024-09-16T00:40:59.584969+0000","last_clean":"2024-09-16T00:40:59.584969+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:40:59.584969+0000","last_undegraded":"2024-09-16T00:40:59.584969+0000","last_fullsized":"2024-09-16T00:40:59.584969+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588578+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:40:59.588578+0000","last_peered":"2024-09-16T00:40:59.588578+0000","last_clean":"2024-09-16T00:40:59.588578+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:40:59.588578+0000","last_undegraded":"2024-09-16T00:40:59.588578+0000","last_fullsized":"2024-09-16T00:40:59.588578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747585+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:40:59.747585+0000","last_peered":"2024-09-16T00:40:59.747585+0000","last_clean":"2024-09-16T00:40:59.747585+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:40:59.747585+0000","last_undegraded":"2024-09-16T00:40:59.747585+0000","last_fullsized":"2024-09-16T00:40:59.747585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747194+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:40:59.747194+0000","last_peered":"2024-09-16T00:40:59.747194+0000","last_clean":"2024-09-16T00:40:59.747194+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:40:59.747194+0000","last_undegraded":"2024-09-16T00:40:59.747194+0000","last_fullsized":"2024-09-16T00:40:59.747194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594858+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:40:59.594858+0000","last_peered":"2024-09-16T00:40:59.594858+0000","last_clean":"2024-09-16T00:40:59.594858+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:40:59.594858+0000","last_undegraded":"2024-09-16T00:40:59.594858+0000","last_fullsized":"2024-09-16T00:40:59.594858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747376+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:40:59.747376+0000","last_peered":"2024-09-16T00:40:59.747376+0000","last_clean":"2024-09-16T00:40:59.747376+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:40:59.747376+0000","last_undegraded":"2024-09-16T00:40:59.747376+0000","last_fullsized":"2024-09-16T00:40:59.747376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588250+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:40:59.588250+0000","last_peered":"2024-09-16T00:40:59.588250+0000","last_clean":"2024-09-16T00:40:59.588250+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:40:59.588250+0000","last_undegraded":"2024-09-16T00:40:59.588250+0000","last_fullsized":"2024-09-16T00:40:59.588250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587684+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:40:59.587684+0000","last_peered":"2024-09-16T00:40:59.587684+0000","last_clean":"2024-09-16T00:40:59.587684+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:40:59.587684+0000","last_undegraded":"2024-09-16T00:40:59.587684+0000","last_fullsized":"2024-09-16T00:40:59.587684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588368+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:40:59.588368+0000","last_peered":"2024-09-16T00:40:59.588368+0000","last_clean":"2024-09-16T00:40:59.588368+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:40:59.588368+0000","last_undegraded":"2024-09-16T00:40:59.588368+0000","last_fullsized":"2024-09-16T00:40:59.588368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586131+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:40:59.586131+0000","last_peered":"2024-09-16T00:40:59.586131+0000","last_clean":"2024-09-16T00:40:59.586131+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:40:59.586131+0000","last_undegraded":"2024-09-16T00:40:59.586131+0000","last_fullsized":"2024-09-16T00:40:59.586131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588144+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:40:59.588144+0000","last_peered":"2024-09-16T00:40:59.588144+0000","last_clean":"2024-09-16T00:40:59.588144+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:40:59.588144+0000","last_undegraded":"2024-09-16T00:40:59.588144+0000","last_fullsized":"2024-09-16T00:40:59.588144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747412+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:40:59.747412+0000","last_peered":"2024-09-16T00:40:59.747412+0000","last_clean":"2024-09-16T00:40:59.747412+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:40:59.747412+0000","last_undegraded":"2024-09-16T00:40:59.747412+0000","last_fullsized":"2024-09-16T00:40:59.747412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588676+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:40:59.588676+0000","last_peered":"2024-09-16T00:40:59.588676+0000","last_clean":"2024-09-16T00:40:59.588676+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:40:59.588676+0000","last_undegraded":"2024-09-16T00:40:59.588676+0000","last_fullsized":"2024-09-16T00:40:59.588676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587600+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:40:59.587600+0000","last_peered":"2024-09-16T00:40:59.587600+0000","last_clean":"2024-09-16T00:40:59.587600+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:40:59.587600+0000","last_undegraded":"2024-09-16T00:40:59.587600+0000","last_fullsized":"2024-09-16T00:40:59.587600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162004+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:41:00.162004+0000","last_peered":"2024-09-16T00:41:00.162004+0000","last_clean":"2024-09-16T00:41:00.162004+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:41:00.162004+0000","last_undegraded":"2024-09-16T00:41:00.162004+0000","last_fullsized":"2024-09-16T00:41:00.162004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595476+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:40:59.595476+0000","last_peered":"2024-09-16T00:40:59.595476+0000","last_clean":"2024-09-16T00:40:59.595476+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:40:59.595476+0000","last_undegraded":"2024-09-16T00:40:59.595476+0000","last_fullsized":"2024-09-16T00:40:59.595476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747786+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:40:59.747786+0000","last_peered":"2024-09-16T00:40:59.747786+0000","last_clean":"2024-09-16T00:40:59.747786+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:40:59.747786+0000","last_undegraded":"2024-09-16T00:40:59.747786+0000","last_fullsized":"2024-09-16T00:40:59.747786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588960+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:40:59.588960+0000","last_peered":"2024-09-16T00:40:59.588960+0000","last_clean":"2024-09-16T00:40:59.588960+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:40:59.588960+0000","last_undegraded":"2024-09-16T00:40:59.588960+0000","last_fullsized":"2024-09-16T00:40:59.588960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.748038+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:40:59.748038+0000","last_peered":"2024-09-16T00:40:59.748038+0000","last_clean":"2024-09-16T00:40:59.748038+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:40:59.748038+0000","last_undegraded":"2024-09-16T00:40:59.748038+0000","last_fullsized":"2024-09-16T00:40:59.748038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587358+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:40:59.587358+0000","last_peered":"2024-09-16T00:40:59.587358+0000","last_clean":"2024-09-16T00:40:59.587358+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:40:59.587358+0000","last_undegraded":"2024-09-16T00:40:59.587358+0000","last_fullsized":"2024-09-16T00:40:59.587358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747240+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:40:59.747240+0000","last_peered":"2024-09-16T00:40:59.747240+0000","last_clean":"2024-09-16T00:40:59.747240+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:40:59.747240+0000","last_undegraded":"2024-09-16T00:40:59.747240+0000","last_fullsized":"2024-09-16T00:40:59.747240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587485+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:40:59.587485+0000","last_peered":"2024-09-16T00:40:59.587485+0000","last_clean":"2024-09-16T00:40:59.587485+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:40:59.587485+0000","last_undegraded":"2024-09-16T00:40:59.587485+0000","last_fullsized":"2024-09-16T00:40:59.587485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595614+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:40:59.595614+0000","last_peered":"2024-09-16T00:40:59.595614+0000","last_clean":"2024-09-16T00:40:59.595614+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:40:59.595614+0000","last_undegraded":"2024-09-16T00:40:59.595614+0000","last_fullsized":"2024-09-16T00:40:59.595614+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747469+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:40:59.747469+0000","last_peered":"2024-09-16T00:40:59.747469+0000","last_clean":"2024-09-16T00:40:59.747469+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:40:59.747469+0000","last_undegraded":"2024-09-16T00:40:59.747469+0000","last_fullsized":"2024-09-16T00:40:59.747469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588263+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:40:59.588263+0000","last_peered":"2024-09-16T00:40:59.588263+0000","last_clean":"2024-09-16T00:40:59.588263+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:40:59.588263+0000","last_undegraded":"2024-09-16T00:40:59.588263+0000","last_fullsized":"2024-09-16T00:40:59.588263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586045+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:40:59.586045+0000","last_peered":"2024-09-16T00:40:59.586045+0000","last_clean":"2024-09-16T00:40:59.586045+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:40:59.586045+0000","last_undegraded":"2024-09-16T00:40:59.586045+0000","last_fullsized":"2024-09-16T00:40:59.586045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747774+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:40:59.747774+0000","last_peered":"2024-09-16T00:40:59.747774+0000","last_clean":"2024-09-16T00:40:59.747774+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:40:59.747774+0000","last_undegraded":"2024-09-16T00:40:59.747774+0000","last_fullsized":"2024-09-16T00:40:59.747774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587142+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:40:59.587142+0000","last_peered":"2024-09-16T00:40:59.587142+0000","last_clean":"2024-09-16T00:40:59.587142+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:40:59.587142+0000","last_undegraded":"2024-09-16T00:40:59.587142+0000","last_fullsized":"2024-09-16T00:40:59.587142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161639+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:41:00.161639+0000","last_peered":"2024-09-16T00:41:00.161639+0000","last_clean":"2024-09-16T00:41:00.161639+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:41:00.161639+0000","last_undegraded":"2024-09-16T00:41:00.161639+0000","last_fullsized":"2024-09-16T00:41:00.161639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161754+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:41:00.161754+0000","last_peered":"2024-09-16T00:41:00.161754+0000","last_clean":"2024-09-16T00:41:00.161754+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:41:00.161754+0000","last_undegraded":"2024-09-16T00:41:00.161754+0000","last_fullsized":"2024-09-16T00:41:00.161754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595267+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:40:59.595267+0000","last_peered":"2024-09-16T00:40:59.595267+0000","last_clean":"2024-09-16T00:40:59.595267+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:40:59.595267+0000","last_undegraded":"2024-09-16T00:40:59.595267+0000","last_fullsized":"2024-09-16T00:40:59.595267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590469+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:40:59.590469+0000","last_peered":"2024-09-16T00:40:59.590469+0000","last_clean":"2024-09-16T00:40:59.590469+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:40:59.590469+0000","last_undegraded":"2024-09-16T00:40:59.590469+0000","last_fullsized":"2024-09-16T00:40:59.590469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592840+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:40:59.592840+0000","last_peered":"2024-09-16T00:40:59.592840+0000","last_clean":"2024-09-16T00:40:59.592840+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:40:59.592840+0000","last_undegraded":"2024-09-16T00:40:59.592840+0000","last_fullsized":"2024-09-16T00:40:59.592840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747793+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:40:59.747793+0000","last_peered":"2024-09-16T00:40:59.747793+0000","last_clean":"2024-09-16T00:40:59.747793+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:40:59.747793+0000","last_undegraded":"2024-09-16T00:40:59.747793+0000","last_fullsized":"2024-09-16T00:40:59.747793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587933+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:40:59.587933+0000","last_peered":"2024-09-16T00:40:59.587933+0000","last_clean":"2024-09-16T00:40:59.587933+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:40:59.587933+0000","last_undegraded":"2024-09-16T00:40:59.587933+0000","last_fullsized":"2024-09-16T00:40:59.587933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747138+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:40:59.747138+0000","last_peered":"2024-09-16T00:40:59.747138+0000","last_clean":"2024-09-16T00:40:59.747138+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:40:59.747138+0000","last_undegraded":"2024-09-16T00:40:59.747138+0000","last_fullsized":"2024-09-16T00:40:59.747138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592328+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:40:59.592328+0000","last_peered":"2024-09-16T00:40:59.592328+0000","last_clean":"2024-09-16T00:40:59.592328+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:40:59.592328+0000","last_undegraded":"2024-09-16T00:40:59.592328+0000","last_fullsized":"2024-09-16T00:40:59.592328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747364+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:40:59.747364+0000","last_peered":"2024-09-16T00:40:59.747364+0000","last_clean":"2024-09-16T00:40:59.747364+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:40:59.747364+0000","last_undegraded":"2024-09-16T00:40:59.747364+0000","last_fullsized":"2024-09-16T00:40:59.747364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586035+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:40:59.586035+0000","last_peered":"2024-09-16T00:40:59.586035+0000","last_clean":"2024-09-16T00:40:59.586035+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:40:59.586035+0000","last_undegraded":"2024-09-16T00:40:59.586035+0000","last_fullsized":"2024-09-16T00:40:59.586035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590395+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:40:59.590395+0000","last_peered":"2024-09-16T00:40:59.590395+0000","last_clean":"2024-09-16T00:40:59.590395+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:40:59.590395+0000","last_undegraded":"2024-09-16T00:40:59.590395+0000","last_fullsized":"2024-09-16T00:40:59.590395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162106+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:41:00.162106+0000","last_peered":"2024-09-16T00:41:00.162106+0000","last_clean":"2024-09-16T00:41:00.162106+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:41:00.162106+0000","last_undegraded":"2024-09-16T00:41:00.162106+0000","last_fullsized":"2024-09-16T00:41:00.162106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162139+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:41:00.162139+0000","last_peered":"2024-09-16T00:41:00.162139+0000","last_clean":"2024-09-16T00:41:00.162139+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:41:00.162139+0000","last_undegraded":"2024-09-16T00:41:00.162139+0000","last_fullsized":"2024-09-16T00:41:00.162139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747779+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:40:59.747779+0000","last_peered":"2024-09-16T00:40:59.747779+0000","last_clean":"2024-09-16T00:40:59.747779+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:40:59.747779+0000","last_undegraded":"2024-09-16T00:40:59.747779+0000","last_fullsized":"2024-09-16T00:40:59.747779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64866,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":199,"num_read_kb":165,"num_write":91,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":194598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547241,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":518379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776174,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37028,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708124,"statfs":{"total":95995035648,"available":95957118976,"internally_reserved":0,"allocated":1716224,"data_stored":1173009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939698,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":518961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168632,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":518983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397565,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1638400,"data_stored":1113425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":514283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1642496,"data_stored":1108929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":518401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215185,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":514387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444119,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1110016,"data_stored":578385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1105920,"data_stored":578351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":514373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:41:08.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:41:08.174 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:07 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/464174409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.257+0000 7fe18d08d640 1 -- 172.21.15.31:0/3672414033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 msgr2=0x7fe188100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.257+0000 7fe18d08d640 1 --2- 172.21.15.31:0/3672414033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 0x7fe188100e30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe178009900 tx=0x7fe17802a410 comp rx=0 tx=0).stop 2024-09-16T00:41:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.257+0000 7fe18d08d640 1 -- 172.21.15.31:0/3672414033 shutdown_connections 2024-09-16T00:41:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.257+0000 7fe18d08d640 1 --2- 172.21.15.31:0/3672414033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1881025e0 0x7fe188108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.257+0000 7fe18d08d640 1 --2- 172.21.15.31:0/3672414033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe188101c20 0x7fe1881020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.257+0000 7fe18d08d640 1 --2- 172.21.15.31:0/3672414033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 0x7fe188100e30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.257+0000 7fe18d08d640 1 -- 172.21.15.31:0/3672414033 >> 172.21.15.31:0/3672414033 conn(0x7fe1880fc190 msgr2=0x7fe1880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.257+0000 7fe18d08d640 1 -- 172.21.15.31:0/3672414033 shutdown_connections 2024-09-16T00:41:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.257+0000 7fe18d08d640 1 -- 172.21.15.31:0/3672414033 wait complete. 2024-09-16T00:41:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18d08d640 1 Processor -- start 2024-09-16T00:41:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18d08d640 1 -- start start 2024-09-16T00:41:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18d08d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 0x7fe188197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18d08d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe188101c20 0x7fe188198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18d08d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1881025e0 0x7fe18819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18d08d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe18810d7c0 con 0x7fe188100a30 2024-09-16T00:41:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18d08d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe18810d640 con 0x7fe1881025e0 2024-09-16T00:41:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18d08d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe18810d940 con 0x7fe188101c20 2024-09-16T00:41:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe186d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 0x7fe188197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe186d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 0x7fe188197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60318/0 (socket says 172.21.15.31:60318) 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe186d9d640 1 -- 172.21.15.31:0/798242053 learned_addr learned my addr 172.21.15.31:0/798242053 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18759e640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1881025e0 0x7fe18819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe186d9d640 1 -- 172.21.15.31:0/798242053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe188101c20 msgr2=0x7fe188198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18659c640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe188101c20 0x7fe188198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe186d9d640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe188101c20 0x7fe188198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe186d9d640 1 -- 172.21.15.31:0/798242053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1881025e0 msgr2=0x7fe18819f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe186d9d640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1881025e0 0x7fe18819f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe186d9d640 1 -- 172.21.15.31:0/798242053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18819fb00 con 0x7fe188100a30 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18759e640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1881025e0 0x7fe18819f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.258+0000 7fe18659c640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe188101c20 0x7fe188198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.259+0000 7fe186d9d640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 0x7fe188197ee0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe178006010 tx=0x7fe1780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.259+0000 7fe16ffff640 1 -- 172.21.15.31:0/798242053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17803f030 con 0x7fe188100a30 2024-09-16T00:41:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.259+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18819fd90 con 0x7fe188100a30 2024-09-16T00:41:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.259+0000 7fe16ffff640 1 -- 172.21.15.31:0/798242053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe178003ce0 con 0x7fe188100a30 2024-09-16T00:41:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.259+0000 7fe16ffff640 1 -- 172.21.15.31:0/798242053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178035070 con 0x7fe188100a30 2024-09-16T00:41:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.259+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881a01d0 con 0x7fe188100a30 2024-09-16T00:41:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.260+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe188107f10 con 0x7fe188100a30 2024-09-16T00:41:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.261+0000 7fe16ffff640 1 -- 172.21.15.31:0/798242053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe178042030 con 0x7fe188100a30 2024-09-16T00:41:08.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.262+0000 7fe16ffff640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1700a3250 0x7fe1700a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:08.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.262+0000 7fe18659c640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1700a3250 0x7fe1700a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:08.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.262+0000 7fe16ffff640 1 -- 172.21.15.31:0/798242053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fe1780f01a0 con 0x7fe188100a30 2024-09-16T00:41:08.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.262+0000 7fe18659c640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1700a3250 0x7fe1700a5710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe1881993f0 tx=0x7fe17c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.266+0000 7fe16ffff640 1 -- 172.21.15.31:0/798242053 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1780bcb10 con 0x7fe188100a30 2024-09-16T00:41:08.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.443+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe188062630 con 0x7fe1700a3250 2024-09-16T00:41:08.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.451+0000 7fe16ffff640 1 -- 172.21.15.31:0/798242053 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340406 (secure 0 0 0) 0x7fe188062630 con 0x7fe1700a3250 2024-09-16T00:41:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:08.456 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:41:08.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1700a3250 msgr2=0x7fe1700a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1700a3250 0x7fe1700a5710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe1881993f0 tx=0x7fe17c005e50 comp rx=0 tx=0).stop 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 msgr2=0x7fe188197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 0x7fe188197ee0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe178006010 tx=0x7fe1780042f0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 shutdown_connections 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1881025e0 0x7fe18819f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe1700a3250 0x7fe1700a5710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe188101c20 0x7fe188198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 --2- 172.21.15.31:0/798242053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe188100a30 0x7fe188197ee0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 >> 172.21.15.31:0/798242053 conn(0x7fe1880fc190 msgr2=0x7fe1881050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 shutdown_connections 2024-09-16T00:41:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.455+0000 7fe18d08d640 1 -- 172.21.15.31:0/798242053 wait complete. 2024-09-16T00:41:08.466 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-16T00:41:07.685864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655468,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":325,"num_read_kb":274,"num_write":274,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":715,"ondisk_log_size":715,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400920,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540904,"statfs":{"total":1151940427776,"available":1151529885696,"internally_reserved":0,"allocated":14483456,"data_stored":8169866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":557,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.109893"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586150+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:40:59.586150+0000","last_peered":"2024-09-16T00:40:59.586150+0000","last_clean":"2024-09-16T00:40:59.586150+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:40:59.586150+0000","last_undegraded":"2024-09-16T00:40:59.586150+0000","last_fullsized":"2024-09-16T00:40:59.586150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589958+0000","last_change":"2024-09-16T00:40:59.589958+0000","last_active":"2024-09-16T00:40:59.589958+0000","last_peered":"2024-09-16T00:40:59.589958+0000","last_clean":"2024-09-16T00:40:59.589958+0000","last_became_active":"2024-09-16T00:40:59.589549+0000","last_became_peered":"2024-09-16T00:40:59.589549+0000","last_unstale":"2024-09-16T00:40:59.589958+0000","last_undegraded":"2024-09-16T00:40:59.589958+0000","last_fullsized":"2024-09-16T00:40:59.589958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:33.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588133+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:40:59.588133+0000","last_peered":"2024-09-16T00:40:59.588133+0000","last_clean":"2024-09-16T00:40:59.588133+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:40:59.588133+0000","last_undegraded":"2024-09-16T00:40:59.588133+0000","last_fullsized":"2024-09-16T00:40:59.588133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747727+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:40:59.747727+0000","last_peered":"2024-09-16T00:40:59.747727+0000","last_clean":"2024-09-16T00:40:59.747727+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:40:59.747727+0000","last_undegraded":"2024-09-16T00:40:59.747727+0000","last_fullsized":"2024-09-16T00:40:59.747727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162291+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:41:00.162291+0000","last_peered":"2024-09-16T00:41:00.162291+0000","last_clean":"2024-09-16T00:41:00.162291+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:41:00.162291+0000","last_undegraded":"2024-09-16T00:41:00.162291+0000","last_fullsized":"2024-09-16T00:41:00.162291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589935+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:40:59.589935+0000","last_peered":"2024-09-16T00:40:59.589935+0000","last_clean":"2024-09-16T00:40:59.589935+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:40:59.589935+0000","last_undegraded":"2024-09-16T00:40:59.589935+0000","last_fullsized":"2024-09-16T00:40:59.589935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586994+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:40:59.586994+0000","last_peered":"2024-09-16T00:40:59.586994+0000","last_clean":"2024-09-16T00:40:59.586994+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:40:59.586994+0000","last_undegraded":"2024-09-16T00:40:59.586994+0000","last_fullsized":"2024-09-16T00:40:59.586994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587478+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:40:59.587478+0000","last_peered":"2024-09-16T00:40:59.587478+0000","last_clean":"2024-09-16T00:40:59.587478+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:40:59.587478+0000","last_undegraded":"2024-09-16T00:40:59.587478+0000","last_fullsized":"2024-09-16T00:40:59.587478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590172+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:40:59.590172+0000","last_peered":"2024-09-16T00:40:59.590172+0000","last_clean":"2024-09-16T00:40:59.590172+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:40:59.590172+0000","last_undegraded":"2024-09-16T00:40:59.590172+0000","last_fullsized":"2024-09-16T00:40:59.590172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587026+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:40:59.587026+0000","last_peered":"2024-09-16T00:40:59.587026+0000","last_clean":"2024-09-16T00:40:59.587026+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:40:59.587026+0000","last_undegraded":"2024-09-16T00:40:59.587026+0000","last_fullsized":"2024-09-16T00:40:59.587026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.596631+0000","last_change":"2024-09-16T00:40:59.596631+0000","last_active":"2024-09-16T00:40:59.596631+0000","last_peered":"2024-09-16T00:40:59.596631+0000","last_clean":"2024-09-16T00:40:59.596631+0000","last_became_active":"2024-09-16T00:40:59.596165+0000","last_became_peered":"2024-09-16T00:40:59.596165+0000","last_unstale":"2024-09-16T00:40:59.596631+0000","last_undegraded":"2024-09-16T00:40:59.596631+0000","last_fullsized":"2024-09-16T00:40:59.596631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:56.918291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,1],"acting":[9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590072+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:40:59.590072+0000","last_peered":"2024-09-16T00:40:59.590072+0000","last_clean":"2024-09-16T00:40:59.590072+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:40:59.590072+0000","last_undegraded":"2024-09-16T00:40:59.590072+0000","last_fullsized":"2024-09-16T00:40:59.590072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592530+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:40:59.592530+0000","last_peered":"2024-09-16T00:40:59.592530+0000","last_clean":"2024-09-16T00:40:59.592530+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:40:59.592530+0000","last_undegraded":"2024-09-16T00:40:59.592530+0000","last_fullsized":"2024-09-16T00:40:59.592530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586212+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:40:59.586212+0000","last_peered":"2024-09-16T00:40:59.586212+0000","last_clean":"2024-09-16T00:40:59.586212+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:40:59.586212+0000","last_undegraded":"2024-09-16T00:40:59.586212+0000","last_fullsized":"2024-09-16T00:40:59.586212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590233+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:40:59.590233+0000","last_peered":"2024-09-16T00:40:59.590233+0000","last_clean":"2024-09-16T00:40:59.590233+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:40:59.590233+0000","last_undegraded":"2024-09-16T00:40:59.590233+0000","last_fullsized":"2024-09-16T00:40:59.590233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747322+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:40:59.747322+0000","last_peered":"2024-09-16T00:40:59.747322+0000","last_clean":"2024-09-16T00:40:59.747322+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:40:59.747322+0000","last_undegraded":"2024-09-16T00:40:59.747322+0000","last_fullsized":"2024-09-16T00:40:59.747322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747467+0000","last_change":"2024-09-16T00:39:59.591063+0000","last_active":"2024-09-16T00:40:59.747467+0000","last_peered":"2024-09-16T00:40:59.747467+0000","last_clean":"2024-09-16T00:40:59.747467+0000","last_became_active":"2024-09-16T00:39:59.590574+0000","last_became_peered":"2024-09-16T00:39:59.590574+0000","last_unstale":"2024-09-16T00:40:59.747467+0000","last_undegraded":"2024-09-16T00:40:59.747467+0000","last_fullsized":"2024-09-16T00:40:59.747467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:57.020592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590255+0000","last_change":"2024-09-16T00:39:59.595772+0000","last_active":"2024-09-16T00:40:59.590255+0000","last_peered":"2024-09-16T00:40:59.590255+0000","last_clean":"2024-09-16T00:40:59.590255+0000","last_became_active":"2024-09-16T00:39:59.590772+0000","last_became_peered":"2024-09-16T00:39:59.590772+0000","last_unstale":"2024-09-16T00:40:59.590255+0000","last_undegraded":"2024-09-16T00:40:59.590255+0000","last_fullsized":"2024-09-16T00:40:59.590255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:53.236019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747573+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:40:59.747573+0000","last_peered":"2024-09-16T00:40:59.747573+0000","last_clean":"2024-09-16T00:40:59.747573+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:40:59.747573+0000","last_undegraded":"2024-09-16T00:40:59.747573+0000","last_fullsized":"2024-09-16T00:40:59.747573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747545+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:40:59.747545+0000","last_peered":"2024-09-16T00:40:59.747545+0000","last_clean":"2024-09-16T00:40:59.747545+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:40:59.747545+0000","last_undegraded":"2024-09-16T00:40:59.747545+0000","last_fullsized":"2024-09-16T00:40:59.747545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585840+0000","last_change":"2024-09-16T00:39:59.386827+0000","last_active":"2024-09-16T00:40:59.585840+0000","last_peered":"2024-09-16T00:40:59.585840+0000","last_clean":"2024-09-16T00:40:59.585840+0000","last_became_active":"2024-09-16T00:39:59.386389+0000","last_became_peered":"2024-09-16T00:39:59.386389+0000","last_unstale":"2024-09-16T00:40:59.585840+0000","last_undegraded":"2024-09-16T00:40:59.585840+0000","last_fullsized":"2024-09-16T00:40:59.585840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:12:21.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585940+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:40:59.585940+0000","last_peered":"2024-09-16T00:40:59.585940+0000","last_clean":"2024-09-16T00:40:59.585940+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:40:59.585940+0000","last_undegraded":"2024-09-16T00:40:59.585940+0000","last_fullsized":"2024-09-16T00:40:59.585940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587363+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:40:59.587363+0000","last_peered":"2024-09-16T00:40:59.587363+0000","last_clean":"2024-09-16T00:40:59.587363+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:40:59.587363+0000","last_undegraded":"2024-09-16T00:40:59.587363+0000","last_fullsized":"2024-09-16T00:40:59.587363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590175+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:40:59.590175+0000","last_peered":"2024-09-16T00:40:59.590175+0000","last_clean":"2024-09-16T00:40:59.590175+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:40:59.590175+0000","last_undegraded":"2024-09-16T00:40:59.590175+0000","last_fullsized":"2024-09-16T00:40:59.590175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162114+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:41:00.162114+0000","last_peered":"2024-09-16T00:41:00.162114+0000","last_clean":"2024-09-16T00:41:00.162114+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:41:00.162114+0000","last_undegraded":"2024-09-16T00:41:00.162114+0000","last_fullsized":"2024-09-16T00:41:00.162114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590740+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:40:59.590740+0000","last_peered":"2024-09-16T00:40:59.590740+0000","last_clean":"2024-09-16T00:40:59.590740+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:40:59.590740+0000","last_undegraded":"2024-09-16T00:40:59.590740+0000","last_fullsized":"2024-09-16T00:40:59.590740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588415+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:40:59.588415+0000","last_peered":"2024-09-16T00:40:59.588415+0000","last_clean":"2024-09-16T00:40:59.588415+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:40:59.588415+0000","last_undegraded":"2024-09-16T00:40:59.588415+0000","last_fullsized":"2024-09-16T00:40:59.588415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747653+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:40:59.747653+0000","last_peered":"2024-09-16T00:40:59.747653+0000","last_clean":"2024-09-16T00:40:59.747653+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:40:59.747653+0000","last_undegraded":"2024-09-16T00:40:59.747653+0000","last_fullsized":"2024-09-16T00:40:59.747653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586160+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:40:59.586160+0000","last_peered":"2024-09-16T00:40:59.586160+0000","last_clean":"2024-09-16T00:40:59.586160+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:40:59.586160+0000","last_undegraded":"2024-09-16T00:40:59.586160+0000","last_fullsized":"2024-09-16T00:40:59.586160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747093+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:40:59.747093+0000","last_peered":"2024-09-16T00:40:59.747093+0000","last_clean":"2024-09-16T00:40:59.747093+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:40:59.747093+0000","last_undegraded":"2024-09-16T00:40:59.747093+0000","last_fullsized":"2024-09-16T00:40:59.747093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595527+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:40:59.595527+0000","last_peered":"2024-09-16T00:40:59.595527+0000","last_clean":"2024-09-16T00:40:59.595527+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:40:59.595527+0000","last_undegraded":"2024-09-16T00:40:59.595527+0000","last_fullsized":"2024-09-16T00:40:59.595527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587025+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:40:59.587025+0000","last_peered":"2024-09-16T00:40:59.587025+0000","last_clean":"2024-09-16T00:40:59.587025+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:40:59.587025+0000","last_undegraded":"2024-09-16T00:40:59.587025+0000","last_fullsized":"2024-09-16T00:40:59.587025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747410+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:40:59.747410+0000","last_peered":"2024-09-16T00:40:59.747410+0000","last_clean":"2024-09-16T00:40:59.747410+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:40:59.747410+0000","last_undegraded":"2024-09-16T00:40:59.747410+0000","last_fullsized":"2024-09-16T00:40:59.747410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747586+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:40:59.747586+0000","last_peered":"2024-09-16T00:40:59.747586+0000","last_clean":"2024-09-16T00:40:59.747586+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:40:59.747586+0000","last_undegraded":"2024-09-16T00:40:59.747586+0000","last_fullsized":"2024-09-16T00:40:59.747586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585296+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:40:59.585296+0000","last_peered":"2024-09-16T00:40:59.585296+0000","last_clean":"2024-09-16T00:40:59.585296+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:40:59.585296+0000","last_undegraded":"2024-09-16T00:40:59.585296+0000","last_fullsized":"2024-09-16T00:40:59.585296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586000+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:40:59.586000+0000","last_peered":"2024-09-16T00:40:59.586000+0000","last_clean":"2024-09-16T00:40:59.586000+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:40:59.586000+0000","last_undegraded":"2024-09-16T00:40:59.586000+0000","last_fullsized":"2024-09-16T00:40:59.586000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747487+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:40:59.747487+0000","last_peered":"2024-09-16T00:40:59.747487+0000","last_clean":"2024-09-16T00:40:59.747487+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:40:59.747487+0000","last_undegraded":"2024-09-16T00:40:59.747487+0000","last_fullsized":"2024-09-16T00:40:59.747487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592602+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:40:59.592602+0000","last_peered":"2024-09-16T00:40:59.592602+0000","last_clean":"2024-09-16T00:40:59.592602+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:40:59.592602+0000","last_undegraded":"2024-09-16T00:40:59.592602+0000","last_fullsized":"2024-09-16T00:40:59.592602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595212+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:40:59.595212+0000","last_peered":"2024-09-16T00:40:59.595212+0000","last_clean":"2024-09-16T00:40:59.595212+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:40:59.595212+0000","last_undegraded":"2024-09-16T00:40:59.595212+0000","last_fullsized":"2024-09-16T00:40:59.595212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.591482+0000","last_change":"2024-09-16T00:40:59.591482+0000","last_active":"2024-09-16T00:40:59.591482+0000","last_peered":"2024-09-16T00:40:59.591482+0000","last_clean":"2024-09-16T00:40:59.591482+0000","last_became_active":"2024-09-16T00:40:59.590978+0000","last_became_peered":"2024-09-16T00:40:59.590978+0000","last_unstale":"2024-09-16T00:40:59.591482+0000","last_undegraded":"2024-09-16T00:40:59.591482+0000","last_fullsized":"2024-09-16T00:40:59.591482+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:56.975117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594993+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:40:59.594993+0000","last_peered":"2024-09-16T00:40:59.594993+0000","last_clean":"2024-09-16T00:40:59.594993+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:40:59.594993+0000","last_undegraded":"2024-09-16T00:40:59.594993+0000","last_fullsized":"2024-09-16T00:40:59.594993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595325+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:40:59.595325+0000","last_peered":"2024-09-16T00:40:59.595325+0000","last_clean":"2024-09-16T00:40:59.595325+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:40:59.595325+0000","last_undegraded":"2024-09-16T00:40:59.595325+0000","last_fullsized":"2024-09-16T00:40:59.595325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594855+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:40:59.594855+0000","last_peered":"2024-09-16T00:40:59.594855+0000","last_clean":"2024-09-16T00:40:59.594855+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:40:59.594855+0000","last_undegraded":"2024-09-16T00:40:59.594855+0000","last_fullsized":"2024-09-16T00:40:59.594855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747209+0000","last_change":"2024-09-16T00:39:59.376963+0000","last_active":"2024-09-16T00:40:59.747209+0000","last_peered":"2024-09-16T00:40:59.747209+0000","last_clean":"2024-09-16T00:40:59.747209+0000","last_became_active":"2024-09-16T00:39:59.376538+0000","last_became_peered":"2024-09-16T00:39:59.376538+0000","last_unstale":"2024-09-16T00:40:59.747209+0000","last_undegraded":"2024-09-16T00:40:59.747209+0000","last_fullsized":"2024-09-16T00:40:59.747209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:02.077749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587857+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:40:59.587857+0000","last_peered":"2024-09-16T00:40:59.587857+0000","last_clean":"2024-09-16T00:40:59.587857+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:40:59.587857+0000","last_undegraded":"2024-09-16T00:40:59.587857+0000","last_fullsized":"2024-09-16T00:40:59.587857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747729+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:40:59.747729+0000","last_peered":"2024-09-16T00:40:59.747729+0000","last_clean":"2024-09-16T00:40:59.747729+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:40:59.747729+0000","last_undegraded":"2024-09-16T00:40:59.747729+0000","last_fullsized":"2024-09-16T00:40:59.747729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590591+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:40:59.590591+0000","last_peered":"2024-09-16T00:40:59.590591+0000","last_clean":"2024-09-16T00:40:59.590591+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:40:59.590591+0000","last_undegraded":"2024-09-16T00:40:59.590591+0000","last_fullsized":"2024-09-16T00:40:59.590591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592357+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:40:59.592357+0000","last_peered":"2024-09-16T00:40:59.592357+0000","last_clean":"2024-09-16T00:40:59.592357+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:40:59.592357+0000","last_undegraded":"2024-09-16T00:40:59.592357+0000","last_fullsized":"2024-09-16T00:40:59.592357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592684+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:40:59.592684+0000","last_peered":"2024-09-16T00:40:59.592684+0000","last_clean":"2024-09-16T00:40:59.592684+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:40:59.592684+0000","last_undegraded":"2024-09-16T00:40:59.592684+0000","last_fullsized":"2024-09-16T00:40:59.592684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592846+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:40:59.592846+0000","last_peered":"2024-09-16T00:40:59.592846+0000","last_clean":"2024-09-16T00:40:59.592846+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:40:59.592846+0000","last_undegraded":"2024-09-16T00:40:59.592846+0000","last_fullsized":"2024-09-16T00:40:59.592846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589988+0000","last_change":"2024-09-16T00:39:59.593442+0000","last_active":"2024-09-16T00:40:59.589988+0000","last_peered":"2024-09-16T00:40:59.589988+0000","last_clean":"2024-09-16T00:40:59.589988+0000","last_became_active":"2024-09-16T00:39:59.593039+0000","last_became_peered":"2024-09-16T00:39:59.593039+0000","last_unstale":"2024-09-16T00:40:59.589988+0000","last_undegraded":"2024-09-16T00:40:59.589988+0000","last_fullsized":"2024-09-16T00:40:59.589988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:36.644148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587311+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:40:59.587311+0000","last_peered":"2024-09-16T00:40:59.587311+0000","last_clean":"2024-09-16T00:40:59.587311+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:40:59.587311+0000","last_undegraded":"2024-09-16T00:40:59.587311+0000","last_fullsized":"2024-09-16T00:40:59.587311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592642+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:40:59.592642+0000","last_peered":"2024-09-16T00:40:59.592642+0000","last_clean":"2024-09-16T00:40:59.592642+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:40:59.592642+0000","last_undegraded":"2024-09-16T00:40:59.592642+0000","last_fullsized":"2024-09-16T00:40:59.592642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589435+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:40:59.589435+0000","last_peered":"2024-09-16T00:40:59.589435+0000","last_clean":"2024-09-16T00:40:59.589435+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:40:59.589435+0000","last_undegraded":"2024-09-16T00:40:59.589435+0000","last_fullsized":"2024-09-16T00:40:59.589435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161277+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:41:00.161277+0000","last_peered":"2024-09-16T00:41:00.161277+0000","last_clean":"2024-09-16T00:41:00.161277+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:41:00.161277+0000","last_undegraded":"2024-09-16T00:41:00.161277+0000","last_fullsized":"2024-09-16T00:41:00.161277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592960+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:40:59.592960+0000","last_peered":"2024-09-16T00:40:59.592960+0000","last_clean":"2024-09-16T00:40:59.592960+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:40:59.592960+0000","last_undegraded":"2024-09-16T00:40:59.592960+0000","last_fullsized":"2024-09-16T00:40:59.592960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595479+0000","last_change":"2024-09-16T00:39:59.594994+0000","last_active":"2024-09-16T00:40:59.595479+0000","last_peered":"2024-09-16T00:40:59.595479+0000","last_clean":"2024-09-16T00:40:59.595479+0000","last_became_active":"2024-09-16T00:39:59.590640+0000","last_became_peered":"2024-09-16T00:39:59.590640+0000","last_unstale":"2024-09-16T00:40:59.595479+0000","last_undegraded":"2024-09-16T00:40:59.595479+0000","last_fullsized":"2024-09-16T00:40:59.595479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:21.391494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587189+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:40:59.587189+0000","last_peered":"2024-09-16T00:40:59.587189+0000","last_clean":"2024-09-16T00:40:59.587189+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:40:59.587189+0000","last_undegraded":"2024-09-16T00:40:59.587189+0000","last_fullsized":"2024-09-16T00:40:59.587189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747514+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:40:59.747514+0000","last_peered":"2024-09-16T00:40:59.747514+0000","last_clean":"2024-09-16T00:40:59.747514+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:40:59.747514+0000","last_undegraded":"2024-09-16T00:40:59.747514+0000","last_fullsized":"2024-09-16T00:40:59.747514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586731+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:40:59.586731+0000","last_peered":"2024-09-16T00:40:59.586731+0000","last_clean":"2024-09-16T00:40:59.586731+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:40:59.586731+0000","last_undegraded":"2024-09-16T00:40:59.586731+0000","last_fullsized":"2024-09-16T00:40:59.586731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587479+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:40:59.587479+0000","last_peered":"2024-09-16T00:40:59.587479+0000","last_clean":"2024-09-16T00:40:59.587479+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:40:59.587479+0000","last_undegraded":"2024-09-16T00:40:59.587479+0000","last_fullsized":"2024-09-16T00:40:59.587479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588245+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:40:59.588245+0000","last_peered":"2024-09-16T00:40:59.588245+0000","last_clean":"2024-09-16T00:40:59.588245+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:40:59.588245+0000","last_undegraded":"2024-09-16T00:40:59.588245+0000","last_fullsized":"2024-09-16T00:40:59.588245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585828+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:40:59.585828+0000","last_peered":"2024-09-16T00:40:59.585828+0000","last_clean":"2024-09-16T00:40:59.585828+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:40:59.585828+0000","last_undegraded":"2024-09-16T00:40:59.585828+0000","last_fullsized":"2024-09-16T00:40:59.585828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595364+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:40:59.595364+0000","last_peered":"2024-09-16T00:40:59.595364+0000","last_clean":"2024-09-16T00:40:59.595364+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:40:59.595364+0000","last_undegraded":"2024-09-16T00:40:59.595364+0000","last_fullsized":"2024-09-16T00:40:59.595364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592178+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:40:59.592178+0000","last_peered":"2024-09-16T00:40:59.592178+0000","last_clean":"2024-09-16T00:40:59.592178+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:40:59.592178+0000","last_undegraded":"2024-09-16T00:40:59.592178+0000","last_fullsized":"2024-09-16T00:40:59.592178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590248+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:40:59.590248+0000","last_peered":"2024-09-16T00:40:59.590248+0000","last_clean":"2024-09-16T00:40:59.590248+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:40:59.590248+0000","last_undegraded":"2024-09-16T00:40:59.590248+0000","last_fullsized":"2024-09-16T00:40:59.590248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586021+0000","last_change":"2024-09-16T00:39:59.593568+0000","last_active":"2024-09-16T00:40:59.586021+0000","last_peered":"2024-09-16T00:40:59.586021+0000","last_clean":"2024-09-16T00:40:59.586021+0000","last_became_active":"2024-09-16T00:39:59.593209+0000","last_became_peered":"2024-09-16T00:39:59.593209+0000","last_unstale":"2024-09-16T00:40:59.586021+0000","last_undegraded":"2024-09-16T00:40:59.586021+0000","last_fullsized":"2024-09-16T00:40:59.586021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:56.516773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.596544+0000","last_change":"2024-09-16T00:40:59.596544+0000","last_active":"2024-09-16T00:40:59.596544+0000","last_peered":"2024-09-16T00:40:59.596544+0000","last_clean":"2024-09-16T00:40:59.596544+0000","last_became_active":"2024-09-16T00:40:59.596128+0000","last_became_peered":"2024-09-16T00:40:59.596128+0000","last_unstale":"2024-09-16T00:40:59.596544+0000","last_undegraded":"2024-09-16T00:40:59.596544+0000","last_fullsized":"2024-09-16T00:40:59.596544+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:22.691164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587645+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:40:59.587645+0000","last_peered":"2024-09-16T00:40:59.587645+0000","last_clean":"2024-09-16T00:40:59.587645+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:40:59.587645+0000","last_undegraded":"2024-09-16T00:40:59.587645+0000","last_fullsized":"2024-09-16T00:40:59.587645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747362+0000","last_change":"2024-09-16T00:39:59.380258+0000","last_active":"2024-09-16T00:40:59.747362+0000","last_peered":"2024-09-16T00:40:59.747362+0000","last_clean":"2024-09-16T00:40:59.747362+0000","last_became_active":"2024-09-16T00:39:59.379813+0000","last_became_peered":"2024-09-16T00:39:59.379813+0000","last_unstale":"2024-09-16T00:40:59.747362+0000","last_undegraded":"2024-09-16T00:40:59.747362+0000","last_fullsized":"2024-09-16T00:40:59.747362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:03.110305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595055+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:40:59.595055+0000","last_peered":"2024-09-16T00:40:59.595055+0000","last_clean":"2024-09-16T00:40:59.595055+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:40:59.595055+0000","last_undegraded":"2024-09-16T00:40:59.595055+0000","last_fullsized":"2024-09-16T00:40:59.595055+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588623+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:40:59.588623+0000","last_peered":"2024-09-16T00:40:59.588623+0000","last_clean":"2024-09-16T00:40:59.588623+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:40:59.588623+0000","last_undegraded":"2024-09-16T00:40:59.588623+0000","last_fullsized":"2024-09-16T00:40:59.588623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588982+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:40:59.588982+0000","last_peered":"2024-09-16T00:40:59.588982+0000","last_clean":"2024-09-16T00:40:59.588982+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:40:59.588982+0000","last_undegraded":"2024-09-16T00:40:59.588982+0000","last_fullsized":"2024-09-16T00:40:59.588982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586881+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:40:59.586881+0000","last_peered":"2024-09-16T00:40:59.586881+0000","last_clean":"2024-09-16T00:40:59.586881+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:40:59.586881+0000","last_undegraded":"2024-09-16T00:40:59.586881+0000","last_fullsized":"2024-09-16T00:40:59.586881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":265,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:05.289803+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:41:05.289803+0000","last_peered":"2024-09-16T00:41:05.289803+0000","last_clean":"2024-09-16T00:41:05.289803+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:41:05.289803+0000","last_undegraded":"2024-09-16T00:41:05.289803+0000","last_fullsized":"2024-09-16T00:41:05.289803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64068,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":193,"num_read_kb":159,"num_write":34,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585067+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:40:59.585067+0000","last_peered":"2024-09-16T00:40:59.585067+0000","last_clean":"2024-09-16T00:40:59.585067+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:40:59.585067+0000","last_undegraded":"2024-09-16T00:40:59.585067+0000","last_fullsized":"2024-09-16T00:40:59.585067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587571+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:40:59.587571+0000","last_peered":"2024-09-16T00:40:59.587571+0000","last_clean":"2024-09-16T00:40:59.587571+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:40:59.587571+0000","last_undegraded":"2024-09-16T00:40:59.587571+0000","last_fullsized":"2024-09-16T00:40:59.587571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589244+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:40:59.589244+0000","last_peered":"2024-09-16T00:40:59.589244+0000","last_clean":"2024-09-16T00:40:59.589244+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:40:59.589244+0000","last_undegraded":"2024-09-16T00:40:59.589244+0000","last_fullsized":"2024-09-16T00:40:59.589244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590297+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:40:59.590297+0000","last_peered":"2024-09-16T00:40:59.590297+0000","last_clean":"2024-09-16T00:40:59.590297+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:40:59.590297+0000","last_undegraded":"2024-09-16T00:40:59.590297+0000","last_fullsized":"2024-09-16T00:40:59.590297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162296+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:41:00.162296+0000","last_peered":"2024-09-16T00:41:00.162296+0000","last_clean":"2024-09-16T00:41:00.162296+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:41:00.162296+0000","last_undegraded":"2024-09-16T00:41:00.162296+0000","last_fullsized":"2024-09-16T00:41:00.162296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592831+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:40:59.592831+0000","last_peered":"2024-09-16T00:40:59.592831+0000","last_clean":"2024-09-16T00:40:59.592831+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:40:59.592831+0000","last_undegraded":"2024-09-16T00:40:59.592831+0000","last_fullsized":"2024-09-16T00:40:59.592831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584869+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:40:59.584869+0000","last_peered":"2024-09-16T00:40:59.584869+0000","last_clean":"2024-09-16T00:40:59.584869+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:40:59.584869+0000","last_undegraded":"2024-09-16T00:40:59.584869+0000","last_fullsized":"2024-09-16T00:40:59.584869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587926+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:40:59.587926+0000","last_peered":"2024-09-16T00:40:59.587926+0000","last_clean":"2024-09-16T00:40:59.587926+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:40:59.587926+0000","last_undegraded":"2024-09-16T00:40:59.587926+0000","last_fullsized":"2024-09-16T00:40:59.587926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747739+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:40:59.747739+0000","last_peered":"2024-09-16T00:40:59.747739+0000","last_clean":"2024-09-16T00:40:59.747739+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:40:59.747739+0000","last_undegraded":"2024-09-16T00:40:59.747739+0000","last_fullsized":"2024-09-16T00:40:59.747739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747139+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:40:59.747139+0000","last_peered":"2024-09-16T00:40:59.747139+0000","last_clean":"2024-09-16T00:40:59.747139+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:40:59.747139+0000","last_undegraded":"2024-09-16T00:40:59.747139+0000","last_fullsized":"2024-09-16T00:40:59.747139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161443+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:41:00.161443+0000","last_peered":"2024-09-16T00:41:00.161443+0000","last_clean":"2024-09-16T00:41:00.161443+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:41:00.161443+0000","last_undegraded":"2024-09-16T00:41:00.161443+0000","last_fullsized":"2024-09-16T00:41:00.161443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584969+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:40:59.584969+0000","last_peered":"2024-09-16T00:40:59.584969+0000","last_clean":"2024-09-16T00:40:59.584969+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:40:59.584969+0000","last_undegraded":"2024-09-16T00:40:59.584969+0000","last_fullsized":"2024-09-16T00:40:59.584969+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588578+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:40:59.588578+0000","last_peered":"2024-09-16T00:40:59.588578+0000","last_clean":"2024-09-16T00:40:59.588578+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:40:59.588578+0000","last_undegraded":"2024-09-16T00:40:59.588578+0000","last_fullsized":"2024-09-16T00:40:59.588578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747585+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:40:59.747585+0000","last_peered":"2024-09-16T00:40:59.747585+0000","last_clean":"2024-09-16T00:40:59.747585+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:40:59.747585+0000","last_undegraded":"2024-09-16T00:40:59.747585+0000","last_fullsized":"2024-09-16T00:40:59.747585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747194+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:40:59.747194+0000","last_peered":"2024-09-16T00:40:59.747194+0000","last_clean":"2024-09-16T00:40:59.747194+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:40:59.747194+0000","last_undegraded":"2024-09-16T00:40:59.747194+0000","last_fullsized":"2024-09-16T00:40:59.747194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594858+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:40:59.594858+0000","last_peered":"2024-09-16T00:40:59.594858+0000","last_clean":"2024-09-16T00:40:59.594858+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:40:59.594858+0000","last_undegraded":"2024-09-16T00:40:59.594858+0000","last_fullsized":"2024-09-16T00:40:59.594858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747376+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:40:59.747376+0000","last_peered":"2024-09-16T00:40:59.747376+0000","last_clean":"2024-09-16T00:40:59.747376+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:40:59.747376+0000","last_undegraded":"2024-09-16T00:40:59.747376+0000","last_fullsized":"2024-09-16T00:40:59.747376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588250+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:40:59.588250+0000","last_peered":"2024-09-16T00:40:59.588250+0000","last_clean":"2024-09-16T00:40:59.588250+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:40:59.588250+0000","last_undegraded":"2024-09-16T00:40:59.588250+0000","last_fullsized":"2024-09-16T00:40:59.588250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587684+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:40:59.587684+0000","last_peered":"2024-09-16T00:40:59.587684+0000","last_clean":"2024-09-16T00:40:59.587684+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:40:59.587684+0000","last_undegraded":"2024-09-16T00:40:59.587684+0000","last_fullsized":"2024-09-16T00:40:59.587684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588368+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:40:59.588368+0000","last_peered":"2024-09-16T00:40:59.588368+0000","last_clean":"2024-09-16T00:40:59.588368+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:40:59.588368+0000","last_undegraded":"2024-09-16T00:40:59.588368+0000","last_fullsized":"2024-09-16T00:40:59.588368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586131+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:40:59.586131+0000","last_peered":"2024-09-16T00:40:59.586131+0000","last_clean":"2024-09-16T00:40:59.586131+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:40:59.586131+0000","last_undegraded":"2024-09-16T00:40:59.586131+0000","last_fullsized":"2024-09-16T00:40:59.586131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588144+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:40:59.588144+0000","last_peered":"2024-09-16T00:40:59.588144+0000","last_clean":"2024-09-16T00:40:59.588144+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:40:59.588144+0000","last_undegraded":"2024-09-16T00:40:59.588144+0000","last_fullsized":"2024-09-16T00:40:59.588144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747412+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:40:59.747412+0000","last_peered":"2024-09-16T00:40:59.747412+0000","last_clean":"2024-09-16T00:40:59.747412+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:40:59.747412+0000","last_undegraded":"2024-09-16T00:40:59.747412+0000","last_fullsized":"2024-09-16T00:40:59.747412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588676+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:40:59.588676+0000","last_peered":"2024-09-16T00:40:59.588676+0000","last_clean":"2024-09-16T00:40:59.588676+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:40:59.588676+0000","last_undegraded":"2024-09-16T00:40:59.588676+0000","last_fullsized":"2024-09-16T00:40:59.588676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587600+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:40:59.587600+0000","last_peered":"2024-09-16T00:40:59.587600+0000","last_clean":"2024-09-16T00:40:59.587600+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:40:59.587600+0000","last_undegraded":"2024-09-16T00:40:59.587600+0000","last_fullsized":"2024-09-16T00:40:59.587600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162004+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:41:00.162004+0000","last_peered":"2024-09-16T00:41:00.162004+0000","last_clean":"2024-09-16T00:41:00.162004+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:41:00.162004+0000","last_undegraded":"2024-09-16T00:41:00.162004+0000","last_fullsized":"2024-09-16T00:41:00.162004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595476+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:40:59.595476+0000","last_peered":"2024-09-16T00:40:59.595476+0000","last_clean":"2024-09-16T00:40:59.595476+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:40:59.595476+0000","last_undegraded":"2024-09-16T00:40:59.595476+0000","last_fullsized":"2024-09-16T00:40:59.595476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747786+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:40:59.747786+0000","last_peered":"2024-09-16T00:40:59.747786+0000","last_clean":"2024-09-16T00:40:59.747786+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:40:59.747786+0000","last_undegraded":"2024-09-16T00:40:59.747786+0000","last_fullsized":"2024-09-16T00:40:59.747786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588960+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:40:59.588960+0000","last_peered":"2024-09-16T00:40:59.588960+0000","last_clean":"2024-09-16T00:40:59.588960+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:40:59.588960+0000","last_undegraded":"2024-09-16T00:40:59.588960+0000","last_fullsized":"2024-09-16T00:40:59.588960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.748038+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:40:59.748038+0000","last_peered":"2024-09-16T00:40:59.748038+0000","last_clean":"2024-09-16T00:40:59.748038+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:40:59.748038+0000","last_undegraded":"2024-09-16T00:40:59.748038+0000","last_fullsized":"2024-09-16T00:40:59.748038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587358+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:40:59.587358+0000","last_peered":"2024-09-16T00:40:59.587358+0000","last_clean":"2024-09-16T00:40:59.587358+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:40:59.587358+0000","last_undegraded":"2024-09-16T00:40:59.587358+0000","last_fullsized":"2024-09-16T00:40:59.587358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747240+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:40:59.747240+0000","last_peered":"2024-09-16T00:40:59.747240+0000","last_clean":"2024-09-16T00:40:59.747240+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:40:59.747240+0000","last_undegraded":"2024-09-16T00:40:59.747240+0000","last_fullsized":"2024-09-16T00:40:59.747240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587485+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:40:59.587485+0000","last_peered":"2024-09-16T00:40:59.587485+0000","last_clean":"2024-09-16T00:40:59.587485+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:40:59.587485+0000","last_undegraded":"2024-09-16T00:40:59.587485+0000","last_fullsized":"2024-09-16T00:40:59.587485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595614+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:40:59.595614+0000","last_peered":"2024-09-16T00:40:59.595614+0000","last_clean":"2024-09-16T00:40:59.595614+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:40:59.595614+0000","last_undegraded":"2024-09-16T00:40:59.595614+0000","last_fullsized":"2024-09-16T00:40:59.595614+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747469+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:40:59.747469+0000","last_peered":"2024-09-16T00:40:59.747469+0000","last_clean":"2024-09-16T00:40:59.747469+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:40:59.747469+0000","last_undegraded":"2024-09-16T00:40:59.747469+0000","last_fullsized":"2024-09-16T00:40:59.747469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588263+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:40:59.588263+0000","last_peered":"2024-09-16T00:40:59.588263+0000","last_clean":"2024-09-16T00:40:59.588263+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:40:59.588263+0000","last_undegraded":"2024-09-16T00:40:59.588263+0000","last_fullsized":"2024-09-16T00:40:59.588263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586045+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:40:59.586045+0000","last_peered":"2024-09-16T00:40:59.586045+0000","last_clean":"2024-09-16T00:40:59.586045+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:40:59.586045+0000","last_undegraded":"2024-09-16T00:40:59.586045+0000","last_fullsized":"2024-09-16T00:40:59.586045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747774+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:40:59.747774+0000","last_peered":"2024-09-16T00:40:59.747774+0000","last_clean":"2024-09-16T00:40:59.747774+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:40:59.747774+0000","last_undegraded":"2024-09-16T00:40:59.747774+0000","last_fullsized":"2024-09-16T00:40:59.747774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587142+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:40:59.587142+0000","last_peered":"2024-09-16T00:40:59.587142+0000","last_clean":"2024-09-16T00:40:59.587142+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:40:59.587142+0000","last_undegraded":"2024-09-16T00:40:59.587142+0000","last_fullsized":"2024-09-16T00:40:59.587142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161639+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:41:00.161639+0000","last_peered":"2024-09-16T00:41:00.161639+0000","last_clean":"2024-09-16T00:41:00.161639+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:41:00.161639+0000","last_undegraded":"2024-09-16T00:41:00.161639+0000","last_fullsized":"2024-09-16T00:41:00.161639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161754+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:41:00.161754+0000","last_peered":"2024-09-16T00:41:00.161754+0000","last_clean":"2024-09-16T00:41:00.161754+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:41:00.161754+0000","last_undegraded":"2024-09-16T00:41:00.161754+0000","last_fullsized":"2024-09-16T00:41:00.161754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595267+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:40:59.595267+0000","last_peered":"2024-09-16T00:40:59.595267+0000","last_clean":"2024-09-16T00:40:59.595267+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:40:59.595267+0000","last_undegraded":"2024-09-16T00:40:59.595267+0000","last_fullsized":"2024-09-16T00:40:59.595267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590469+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:40:59.590469+0000","last_peered":"2024-09-16T00:40:59.590469+0000","last_clean":"2024-09-16T00:40:59.590469+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:40:59.590469+0000","last_undegraded":"2024-09-16T00:40:59.590469+0000","last_fullsized":"2024-09-16T00:40:59.590469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592840+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:40:59.592840+0000","last_peered":"2024-09-16T00:40:59.592840+0000","last_clean":"2024-09-16T00:40:59.592840+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:40:59.592840+0000","last_undegraded":"2024-09-16T00:40:59.592840+0000","last_fullsized":"2024-09-16T00:40:59.592840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747793+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:40:59.747793+0000","last_peered":"2024-09-16T00:40:59.747793+0000","last_clean":"2024-09-16T00:40:59.747793+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:40:59.747793+0000","last_undegraded":"2024-09-16T00:40:59.747793+0000","last_fullsized":"2024-09-16T00:40:59.747793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587933+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:40:59.587933+0000","last_peered":"2024-09-16T00:40:59.587933+0000","last_clean":"2024-09-16T00:40:59.587933+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:40:59.587933+0000","last_undegraded":"2024-09-16T00:40:59.587933+0000","last_fullsized":"2024-09-16T00:40:59.587933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747138+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:40:59.747138+0000","last_peered":"2024-09-16T00:40:59.747138+0000","last_clean":"2024-09-16T00:40:59.747138+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:40:59.747138+0000","last_undegraded":"2024-09-16T00:40:59.747138+0000","last_fullsized":"2024-09-16T00:40:59.747138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592328+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:40:59.592328+0000","last_peered":"2024-09-16T00:40:59.592328+0000","last_clean":"2024-09-16T00:40:59.592328+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:40:59.592328+0000","last_undegraded":"2024-09-16T00:40:59.592328+0000","last_fullsized":"2024-09-16T00:40:59.592328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747364+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:40:59.747364+0000","last_peered":"2024-09-16T00:40:59.747364+0000","last_clean":"2024-09-16T00:40:59.747364+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:40:59.747364+0000","last_undegraded":"2024-09-16T00:40:59.747364+0000","last_fullsized":"2024-09-16T00:40:59.747364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586035+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:40:59.586035+0000","last_peered":"2024-09-16T00:40:59.586035+0000","last_clean":"2024-09-16T00:40:59.586035+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:40:59.586035+0000","last_undegraded":"2024-09-16T00:40:59.586035+0000","last_fullsized":"2024-09-16T00:40:59.586035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590395+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:40:59.590395+0000","last_peered":"2024-09-16T00:40:59.590395+0000","last_clean":"2024-09-16T00:40:59.590395+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:40:59.590395+0000","last_undegraded":"2024-09-16T00:40:59.590395+0000","last_fullsized":"2024-09-16T00:40:59.590395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162106+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:41:00.162106+0000","last_peered":"2024-09-16T00:41:00.162106+0000","last_clean":"2024-09-16T00:41:00.162106+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:41:00.162106+0000","last_undegraded":"2024-09-16T00:41:00.162106+0000","last_fullsized":"2024-09-16T00:41:00.162106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162139+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:41:00.162139+0000","last_peered":"2024-09-16T00:41:00.162139+0000","last_clean":"2024-09-16T00:41:00.162139+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:41:00.162139+0000","last_undegraded":"2024-09-16T00:41:00.162139+0000","last_fullsized":"2024-09-16T00:41:00.162139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747779+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:40:59.747779+0000","last_peered":"2024-09-16T00:40:59.747779+0000","last_clean":"2024-09-16T00:40:59.747779+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:40:59.747779+0000","last_undegraded":"2024-09-16T00:40:59.747779+0000","last_fullsized":"2024-09-16T00:40:59.747779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64866,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":199,"num_read_kb":165,"num_write":91,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":194598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547241,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":518379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776174,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37028,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708124,"statfs":{"total":95995035648,"available":95957118976,"internally_reserved":0,"allocated":1716224,"data_stored":1173009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939698,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":518961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168632,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":518983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397565,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1638400,"data_stored":1113425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":514283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1642496,"data_stored":1108929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":518401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215185,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":514387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444119,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1110016,"data_stored":578385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1105920,"data_stored":578351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":514373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:41:08.471 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-16T00:41:08.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:41:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 -- 172.21.15.31:0/1891497394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 msgr2=0x7ff224107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/1891497394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 0x7ff224107310 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff214009900 tx=0x7ff21402a410 comp rx=0 tx=0).stop 2024-09-16T00:41:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 -- 172.21.15.31:0/1891497394 shutdown_connections 2024-09-16T00:41:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/1891497394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff224107850 0x7ff224109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/1891497394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 0x7ff224107310 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/1891497394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2241025f0 0x7ff2241049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 -- 172.21.15.31:0/1891497394 >> 172.21.15.31:0/1891497394 conn(0x7ff2240fc170 msgr2=0x7ff2240fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 -- 172.21.15.31:0/1891497394 shutdown_connections 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 -- 172.21.15.31:0/1891497394 wait complete. 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 Processor -- start 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 -- start start 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2241025f0 0x7ff224197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 0x7ff224198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff224107850 0x7ff22419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff22410d820 con 0x7ff224104f20 2024-09-16T00:41:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff22410d6a0 con 0x7ff224107850 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.611+0000 7ff22c4ba640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff22410d9a0 con 0x7ff2241025f0 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff229a55640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 0x7ff224198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff229a55640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 0x7ff224198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60332/0 (socket says 172.21.15.31:60332) 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff229a55640 1 -- 172.21.15.31:0/3248664226 learned_addr learned my addr 172.21.15.31:0/3248664226 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff22aa57640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff224107850 0x7ff22419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff22a256640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2241025f0 0x7ff224197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff229a55640 1 -- 172.21.15.31:0/3248664226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2241025f0 msgr2=0x7ff224197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff229a55640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2241025f0 0x7ff224197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff229a55640 1 -- 172.21.15.31:0/3248664226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff224107850 msgr2=0x7ff22419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff229a55640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff224107850 0x7ff22419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff229a55640 1 -- 172.21.15.31:0/3248664226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22419fb40 con 0x7ff224104f20 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff22a256640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2241025f0 0x7ff224197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff229a55640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 0x7ff224198460 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff214031610 tx=0x7ff2140043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff21b7fe640 1 -- 172.21.15.31:0/3248664226 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21403f030 con 0x7ff224104f20 2024-09-16T00:41:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff21b7fe640 1 -- 172.21.15.31:0/3248664226 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff214003ce0 con 0x7ff224104f20 2024-09-16T00:41:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22419fdd0 con 0x7ff224104f20 2024-09-16T00:41:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff21b7fe640 1 -- 172.21.15.31:0/3248664226 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214035070 con 0x7ff224104f20 2024-09-16T00:41:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.612+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2241a0280 con 0x7ff224104f20 2024-09-16T00:41:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.613+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2240fdc60 con 0x7ff224104f20 2024-09-16T00:41:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.614+0000 7ff21b7fe640 1 -- 172.21.15.31:0/3248664226 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff214042030 con 0x7ff224104f20 2024-09-16T00:41:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.615+0000 7ff21b7fe640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2100a32d0 0x7ff2100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.615+0000 7ff22a256640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2100a32d0 0x7ff2100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.616+0000 7ff21b7fe640 1 -- 172.21.15.31:0/3248664226 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7ff2140f0260 con 0x7ff224104f20 2024-09-16T00:41:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.616+0000 7ff22a256640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2100a32d0 0x7ff2100a5790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff220005d00 tx=0x7ff220005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.620+0000 7ff21b7fe640 1 -- 172.21.15.31:0/3248664226 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2140bcb70 con 0x7ff224104f20 2024-09-16T00:41:08.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.793+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff224062630 con 0x7ff2100a32d0 2024-09-16T00:41:08.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.801+0000 7ff21b7fe640 1 -- 172.21.15.31:0/3248664226 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340406 (secure 0 0 0) 0x7ff224062630 con 0x7ff2100a32d0 2024-09-16T00:41:08.805 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:08.806 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:41:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2100a32d0 msgr2=0x7ff2100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2100a32d0 0x7ff2100a5790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff220005d00 tx=0x7ff220005ed0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 msgr2=0x7ff224198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 0x7ff224198460 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff214031610 tx=0x7ff2140043b0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 shutdown_connections 2024-09-16T00:41:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff224107850 0x7ff22419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff224104f20 0x7ff224198460 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2100a32d0 0x7ff2100a5790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 --2- 172.21.15.31:0/3248664226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2241025f0 0x7ff224197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:08.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.804+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 >> 172.21.15.31:0/3248664226 conn(0x7ff2240fc170 msgr2=0x7ff224102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:08.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.805+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 shutdown_connections 2024-09-16T00:41:08.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:08.805+0000 7ff22c4ba640 1 -- 172.21.15.31:0/3248664226 wait complete. 2024-09-16T00:41:08.815 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-16T00:41:07.685864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655468,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":325,"num_read_kb":274,"num_write":274,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":715,"ondisk_log_size":715,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400920,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540904,"statfs":{"total":1151940427776,"available":1151529885696,"internally_reserved":0,"allocated":14483456,"data_stored":8169866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":557,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.109893"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586150+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:40:59.586150+0000","last_peered":"2024-09-16T00:40:59.586150+0000","last_clean":"2024-09-16T00:40:59.586150+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:40:59.586150+0000","last_undegraded":"2024-09-16T00:40:59.586150+0000","last_fullsized":"2024-09-16T00:40:59.586150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589958+0000","last_change":"2024-09-16T00:40:59.589958+0000","last_active":"2024-09-16T00:40:59.589958+0000","last_peered":"2024-09-16T00:40:59.589958+0000","last_clean":"2024-09-16T00:40:59.589958+0000","last_became_active":"2024-09-16T00:40:59.589549+0000","last_became_peered":"2024-09-16T00:40:59.589549+0000","last_unstale":"2024-09-16T00:40:59.589958+0000","last_undegraded":"2024-09-16T00:40:59.589958+0000","last_fullsized":"2024-09-16T00:40:59.589958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:33.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588133+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:40:59.588133+0000","last_peered":"2024-09-16T00:40:59.588133+0000","last_clean":"2024-09-16T00:40:59.588133+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:40:59.588133+0000","last_undegraded":"2024-09-16T00:40:59.588133+0000","last_fullsized":"2024-09-16T00:40:59.588133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747727+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:40:59.747727+0000","last_peered":"2024-09-16T00:40:59.747727+0000","last_clean":"2024-09-16T00:40:59.747727+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:40:59.747727+0000","last_undegraded":"2024-09-16T00:40:59.747727+0000","last_fullsized":"2024-09-16T00:40:59.747727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162291+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:41:00.162291+0000","last_peered":"2024-09-16T00:41:00.162291+0000","last_clean":"2024-09-16T00:41:00.162291+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:41:00.162291+0000","last_undegraded":"2024-09-16T00:41:00.162291+0000","last_fullsized":"2024-09-16T00:41:00.162291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589935+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:40:59.589935+0000","last_peered":"2024-09-16T00:40:59.589935+0000","last_clean":"2024-09-16T00:40:59.589935+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:40:59.589935+0000","last_undegraded":"2024-09-16T00:40:59.589935+0000","last_fullsized":"2024-09-16T00:40:59.589935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586994+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:40:59.586994+0000","last_peered":"2024-09-16T00:40:59.586994+0000","last_clean":"2024-09-16T00:40:59.586994+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:40:59.586994+0000","last_undegraded":"2024-09-16T00:40:59.586994+0000","last_fullsized":"2024-09-16T00:40:59.586994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587478+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:40:59.587478+0000","last_peered":"2024-09-16T00:40:59.587478+0000","last_clean":"2024-09-16T00:40:59.587478+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:40:59.587478+0000","last_undegraded":"2024-09-16T00:40:59.587478+0000","last_fullsized":"2024-09-16T00:40:59.587478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590172+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:40:59.590172+0000","last_peered":"2024-09-16T00:40:59.590172+0000","last_clean":"2024-09-16T00:40:59.590172+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:40:59.590172+0000","last_undegraded":"2024-09-16T00:40:59.590172+0000","last_fullsized":"2024-09-16T00:40:59.590172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587026+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:40:59.587026+0000","last_peered":"2024-09-16T00:40:59.587026+0000","last_clean":"2024-09-16T00:40:59.587026+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:40:59.587026+0000","last_undegraded":"2024-09-16T00:40:59.587026+0000","last_fullsized":"2024-09-16T00:40:59.587026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.596631+0000","last_change":"2024-09-16T00:40:59.596631+0000","last_active":"2024-09-16T00:40:59.596631+0000","last_peered":"2024-09-16T00:40:59.596631+0000","last_clean":"2024-09-16T00:40:59.596631+0000","last_became_active":"2024-09-16T00:40:59.596165+0000","last_became_peered":"2024-09-16T00:40:59.596165+0000","last_unstale":"2024-09-16T00:40:59.596631+0000","last_undegraded":"2024-09-16T00:40:59.596631+0000","last_fullsized":"2024-09-16T00:40:59.596631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:56.918291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,1],"acting":[9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590072+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:40:59.590072+0000","last_peered":"2024-09-16T00:40:59.590072+0000","last_clean":"2024-09-16T00:40:59.590072+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:40:59.590072+0000","last_undegraded":"2024-09-16T00:40:59.590072+0000","last_fullsized":"2024-09-16T00:40:59.590072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592530+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:40:59.592530+0000","last_peered":"2024-09-16T00:40:59.592530+0000","last_clean":"2024-09-16T00:40:59.592530+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:40:59.592530+0000","last_undegraded":"2024-09-16T00:40:59.592530+0000","last_fullsized":"2024-09-16T00:40:59.592530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586212+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:40:59.586212+0000","last_peered":"2024-09-16T00:40:59.586212+0000","last_clean":"2024-09-16T00:40:59.586212+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:40:59.586212+0000","last_undegraded":"2024-09-16T00:40:59.586212+0000","last_fullsized":"2024-09-16T00:40:59.586212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590233+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:40:59.590233+0000","last_peered":"2024-09-16T00:40:59.590233+0000","last_clean":"2024-09-16T00:40:59.590233+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:40:59.590233+0000","last_undegraded":"2024-09-16T00:40:59.590233+0000","last_fullsized":"2024-09-16T00:40:59.590233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747322+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:40:59.747322+0000","last_peered":"2024-09-16T00:40:59.747322+0000","last_clean":"2024-09-16T00:40:59.747322+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:40:59.747322+0000","last_undegraded":"2024-09-16T00:40:59.747322+0000","last_fullsized":"2024-09-16T00:40:59.747322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747467+0000","last_change":"2024-09-16T00:39:59.591063+0000","last_active":"2024-09-16T00:40:59.747467+0000","last_peered":"2024-09-16T00:40:59.747467+0000","last_clean":"2024-09-16T00:40:59.747467+0000","last_became_active":"2024-09-16T00:39:59.590574+0000","last_became_peered":"2024-09-16T00:39:59.590574+0000","last_unstale":"2024-09-16T00:40:59.747467+0000","last_undegraded":"2024-09-16T00:40:59.747467+0000","last_fullsized":"2024-09-16T00:40:59.747467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:57.020592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590255+0000","last_change":"2024-09-16T00:39:59.595772+0000","last_active":"2024-09-16T00:40:59.590255+0000","last_peered":"2024-09-16T00:40:59.590255+0000","last_clean":"2024-09-16T00:40:59.590255+0000","last_became_active":"2024-09-16T00:39:59.590772+0000","last_became_peered":"2024-09-16T00:39:59.590772+0000","last_unstale":"2024-09-16T00:40:59.590255+0000","last_undegraded":"2024-09-16T00:40:59.590255+0000","last_fullsized":"2024-09-16T00:40:59.590255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:53.236019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747573+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:40:59.747573+0000","last_peered":"2024-09-16T00:40:59.747573+0000","last_clean":"2024-09-16T00:40:59.747573+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:40:59.747573+0000","last_undegraded":"2024-09-16T00:40:59.747573+0000","last_fullsized":"2024-09-16T00:40:59.747573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747545+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:40:59.747545+0000","last_peered":"2024-09-16T00:40:59.747545+0000","last_clean":"2024-09-16T00:40:59.747545+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:40:59.747545+0000","last_undegraded":"2024-09-16T00:40:59.747545+0000","last_fullsized":"2024-09-16T00:40:59.747545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585840+0000","last_change":"2024-09-16T00:39:59.386827+0000","last_active":"2024-09-16T00:40:59.585840+0000","last_peered":"2024-09-16T00:40:59.585840+0000","last_clean":"2024-09-16T00:40:59.585840+0000","last_became_active":"2024-09-16T00:39:59.386389+0000","last_became_peered":"2024-09-16T00:39:59.386389+0000","last_unstale":"2024-09-16T00:40:59.585840+0000","last_undegraded":"2024-09-16T00:40:59.585840+0000","last_fullsized":"2024-09-16T00:40:59.585840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:12:21.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585940+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:40:59.585940+0000","last_peered":"2024-09-16T00:40:59.585940+0000","last_clean":"2024-09-16T00:40:59.585940+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:40:59.585940+0000","last_undegraded":"2024-09-16T00:40:59.585940+0000","last_fullsized":"2024-09-16T00:40:59.585940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587363+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:40:59.587363+0000","last_peered":"2024-09-16T00:40:59.587363+0000","last_clean":"2024-09-16T00:40:59.587363+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:40:59.587363+0000","last_undegraded":"2024-09-16T00:40:59.587363+0000","last_fullsized":"2024-09-16T00:40:59.587363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590175+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:40:59.590175+0000","last_peered":"2024-09-16T00:40:59.590175+0000","last_clean":"2024-09-16T00:40:59.590175+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:40:59.590175+0000","last_undegraded":"2024-09-16T00:40:59.590175+0000","last_fullsized":"2024-09-16T00:40:59.590175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162114+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:41:00.162114+0000","last_peered":"2024-09-16T00:41:00.162114+0000","last_clean":"2024-09-16T00:41:00.162114+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:41:00.162114+0000","last_undegraded":"2024-09-16T00:41:00.162114+0000","last_fullsized":"2024-09-16T00:41:00.162114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590740+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:40:59.590740+0000","last_peered":"2024-09-16T00:40:59.590740+0000","last_clean":"2024-09-16T00:40:59.590740+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:40:59.590740+0000","last_undegraded":"2024-09-16T00:40:59.590740+0000","last_fullsized":"2024-09-16T00:40:59.590740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588415+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:40:59.588415+0000","last_peered":"2024-09-16T00:40:59.588415+0000","last_clean":"2024-09-16T00:40:59.588415+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:40:59.588415+0000","last_undegraded":"2024-09-16T00:40:59.588415+0000","last_fullsized":"2024-09-16T00:40:59.588415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747653+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:40:59.747653+0000","last_peered":"2024-09-16T00:40:59.747653+0000","last_clean":"2024-09-16T00:40:59.747653+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:40:59.747653+0000","last_undegraded":"2024-09-16T00:40:59.747653+0000","last_fullsized":"2024-09-16T00:40:59.747653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586160+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:40:59.586160+0000","last_peered":"2024-09-16T00:40:59.586160+0000","last_clean":"2024-09-16T00:40:59.586160+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:40:59.586160+0000","last_undegraded":"2024-09-16T00:40:59.586160+0000","last_fullsized":"2024-09-16T00:40:59.586160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747093+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:40:59.747093+0000","last_peered":"2024-09-16T00:40:59.747093+0000","last_clean":"2024-09-16T00:40:59.747093+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:40:59.747093+0000","last_undegraded":"2024-09-16T00:40:59.747093+0000","last_fullsized":"2024-09-16T00:40:59.747093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595527+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:40:59.595527+0000","last_peered":"2024-09-16T00:40:59.595527+0000","last_clean":"2024-09-16T00:40:59.595527+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:40:59.595527+0000","last_undegraded":"2024-09-16T00:40:59.595527+0000","last_fullsized":"2024-09-16T00:40:59.595527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587025+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:40:59.587025+0000","last_peered":"2024-09-16T00:40:59.587025+0000","last_clean":"2024-09-16T00:40:59.587025+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:40:59.587025+0000","last_undegraded":"2024-09-16T00:40:59.587025+0000","last_fullsized":"2024-09-16T00:40:59.587025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747410+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:40:59.747410+0000","last_peered":"2024-09-16T00:40:59.747410+0000","last_clean":"2024-09-16T00:40:59.747410+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:40:59.747410+0000","last_undegraded":"2024-09-16T00:40:59.747410+0000","last_fullsized":"2024-09-16T00:40:59.747410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747586+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:40:59.747586+0000","last_peered":"2024-09-16T00:40:59.747586+0000","last_clean":"2024-09-16T00:40:59.747586+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:40:59.747586+0000","last_undegraded":"2024-09-16T00:40:59.747586+0000","last_fullsized":"2024-09-16T00:40:59.747586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585296+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:40:59.585296+0000","last_peered":"2024-09-16T00:40:59.585296+0000","last_clean":"2024-09-16T00:40:59.585296+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:40:59.585296+0000","last_undegraded":"2024-09-16T00:40:59.585296+0000","last_fullsized":"2024-09-16T00:40:59.585296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586000+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:40:59.586000+0000","last_peered":"2024-09-16T00:40:59.586000+0000","last_clean":"2024-09-16T00:40:59.586000+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:40:59.586000+0000","last_undegraded":"2024-09-16T00:40:59.586000+0000","last_fullsized":"2024-09-16T00:40:59.586000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747487+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:40:59.747487+0000","last_peered":"2024-09-16T00:40:59.747487+0000","last_clean":"2024-09-16T00:40:59.747487+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:40:59.747487+0000","last_undegraded":"2024-09-16T00:40:59.747487+0000","last_fullsized":"2024-09-16T00:40:59.747487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592602+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:40:59.592602+0000","last_peered":"2024-09-16T00:40:59.592602+0000","last_clean":"2024-09-16T00:40:59.592602+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:40:59.592602+0000","last_undegraded":"2024-09-16T00:40:59.592602+0000","last_fullsized":"2024-09-16T00:40:59.592602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595212+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:40:59.595212+0000","last_peered":"2024-09-16T00:40:59.595212+0000","last_clean":"2024-09-16T00:40:59.595212+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:40:59.595212+0000","last_undegraded":"2024-09-16T00:40:59.595212+0000","last_fullsized":"2024-09-16T00:40:59.595212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.591482+0000","last_change":"2024-09-16T00:40:59.591482+0000","last_active":"2024-09-16T00:40:59.591482+0000","last_peered":"2024-09-16T00:40:59.591482+0000","last_clean":"2024-09-16T00:40:59.591482+0000","last_became_active":"2024-09-16T00:40:59.590978+0000","last_became_peered":"2024-09-16T00:40:59.590978+0000","last_unstale":"2024-09-16T00:40:59.591482+0000","last_undegraded":"2024-09-16T00:40:59.591482+0000","last_fullsized":"2024-09-16T00:40:59.591482+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:56.975117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594993+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:40:59.594993+0000","last_peered":"2024-09-16T00:40:59.594993+0000","last_clean":"2024-09-16T00:40:59.594993+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:40:59.594993+0000","last_undegraded":"2024-09-16T00:40:59.594993+0000","last_fullsized":"2024-09-16T00:40:59.594993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595325+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:40:59.595325+0000","last_peered":"2024-09-16T00:40:59.595325+0000","last_clean":"2024-09-16T00:40:59.595325+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:40:59.595325+0000","last_undegraded":"2024-09-16T00:40:59.595325+0000","last_fullsized":"2024-09-16T00:40:59.595325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594855+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:40:59.594855+0000","last_peered":"2024-09-16T00:40:59.594855+0000","last_clean":"2024-09-16T00:40:59.594855+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:40:59.594855+0000","last_undegraded":"2024-09-16T00:40:59.594855+0000","last_fullsized":"2024-09-16T00:40:59.594855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747209+0000","last_change":"2024-09-16T00:39:59.376963+0000","last_active":"2024-09-16T00:40:59.747209+0000","last_peered":"2024-09-16T00:40:59.747209+0000","last_clean":"2024-09-16T00:40:59.747209+0000","last_became_active":"2024-09-16T00:39:59.376538+0000","last_became_peered":"2024-09-16T00:39:59.376538+0000","last_unstale":"2024-09-16T00:40:59.747209+0000","last_undegraded":"2024-09-16T00:40:59.747209+0000","last_fullsized":"2024-09-16T00:40:59.747209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:02.077749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587857+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:40:59.587857+0000","last_peered":"2024-09-16T00:40:59.587857+0000","last_clean":"2024-09-16T00:40:59.587857+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:40:59.587857+0000","last_undegraded":"2024-09-16T00:40:59.587857+0000","last_fullsized":"2024-09-16T00:40:59.587857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747729+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:40:59.747729+0000","last_peered":"2024-09-16T00:40:59.747729+0000","last_clean":"2024-09-16T00:40:59.747729+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:40:59.747729+0000","last_undegraded":"2024-09-16T00:40:59.747729+0000","last_fullsized":"2024-09-16T00:40:59.747729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590591+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:40:59.590591+0000","last_peered":"2024-09-16T00:40:59.590591+0000","last_clean":"2024-09-16T00:40:59.590591+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:40:59.590591+0000","last_undegraded":"2024-09-16T00:40:59.590591+0000","last_fullsized":"2024-09-16T00:40:59.590591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592357+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:40:59.592357+0000","last_peered":"2024-09-16T00:40:59.592357+0000","last_clean":"2024-09-16T00:40:59.592357+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:40:59.592357+0000","last_undegraded":"2024-09-16T00:40:59.592357+0000","last_fullsized":"2024-09-16T00:40:59.592357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592684+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:40:59.592684+0000","last_peered":"2024-09-16T00:40:59.592684+0000","last_clean":"2024-09-16T00:40:59.592684+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:40:59.592684+0000","last_undegraded":"2024-09-16T00:40:59.592684+0000","last_fullsized":"2024-09-16T00:40:59.592684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592846+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:40:59.592846+0000","last_peered":"2024-09-16T00:40:59.592846+0000","last_clean":"2024-09-16T00:40:59.592846+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:40:59.592846+0000","last_undegraded":"2024-09-16T00:40:59.592846+0000","last_fullsized":"2024-09-16T00:40:59.592846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589988+0000","last_change":"2024-09-16T00:39:59.593442+0000","last_active":"2024-09-16T00:40:59.589988+0000","last_peered":"2024-09-16T00:40:59.589988+0000","last_clean":"2024-09-16T00:40:59.589988+0000","last_became_active":"2024-09-16T00:39:59.593039+0000","last_became_peered":"2024-09-16T00:39:59.593039+0000","last_unstale":"2024-09-16T00:40:59.589988+0000","last_undegraded":"2024-09-16T00:40:59.589988+0000","last_fullsized":"2024-09-16T00:40:59.589988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:36.644148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587311+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:40:59.587311+0000","last_peered":"2024-09-16T00:40:59.587311+0000","last_clean":"2024-09-16T00:40:59.587311+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:40:59.587311+0000","last_undegraded":"2024-09-16T00:40:59.587311+0000","last_fullsized":"2024-09-16T00:40:59.587311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592642+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:40:59.592642+0000","last_peered":"2024-09-16T00:40:59.592642+0000","last_clean":"2024-09-16T00:40:59.592642+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:40:59.592642+0000","last_undegraded":"2024-09-16T00:40:59.592642+0000","last_fullsized":"2024-09-16T00:40:59.592642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589435+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:40:59.589435+0000","last_peered":"2024-09-16T00:40:59.589435+0000","last_clean":"2024-09-16T00:40:59.589435+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:40:59.589435+0000","last_undegraded":"2024-09-16T00:40:59.589435+0000","last_fullsized":"2024-09-16T00:40:59.589435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161277+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:41:00.161277+0000","last_peered":"2024-09-16T00:41:00.161277+0000","last_clean":"2024-09-16T00:41:00.161277+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:41:00.161277+0000","last_undegraded":"2024-09-16T00:41:00.161277+0000","last_fullsized":"2024-09-16T00:41:00.161277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592960+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:40:59.592960+0000","last_peered":"2024-09-16T00:40:59.592960+0000","last_clean":"2024-09-16T00:40:59.592960+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:40:59.592960+0000","last_undegraded":"2024-09-16T00:40:59.592960+0000","last_fullsized":"2024-09-16T00:40:59.592960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595479+0000","last_change":"2024-09-16T00:39:59.594994+0000","last_active":"2024-09-16T00:40:59.595479+0000","last_peered":"2024-09-16T00:40:59.595479+0000","last_clean":"2024-09-16T00:40:59.595479+0000","last_became_active":"2024-09-16T00:39:59.590640+0000","last_became_peered":"2024-09-16T00:39:59.590640+0000","last_unstale":"2024-09-16T00:40:59.595479+0000","last_undegraded":"2024-09-16T00:40:59.595479+0000","last_fullsized":"2024-09-16T00:40:59.595479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:21.391494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587189+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:40:59.587189+0000","last_peered":"2024-09-16T00:40:59.587189+0000","last_clean":"2024-09-16T00:40:59.587189+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:40:59.587189+0000","last_undegraded":"2024-09-16T00:40:59.587189+0000","last_fullsized":"2024-09-16T00:40:59.587189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747514+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:40:59.747514+0000","last_peered":"2024-09-16T00:40:59.747514+0000","last_clean":"2024-09-16T00:40:59.747514+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:40:59.747514+0000","last_undegraded":"2024-09-16T00:40:59.747514+0000","last_fullsized":"2024-09-16T00:40:59.747514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586731+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:40:59.586731+0000","last_peered":"2024-09-16T00:40:59.586731+0000","last_clean":"2024-09-16T00:40:59.586731+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:40:59.586731+0000","last_undegraded":"2024-09-16T00:40:59.586731+0000","last_fullsized":"2024-09-16T00:40:59.586731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587479+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:40:59.587479+0000","last_peered":"2024-09-16T00:40:59.587479+0000","last_clean":"2024-09-16T00:40:59.587479+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:40:59.587479+0000","last_undegraded":"2024-09-16T00:40:59.587479+0000","last_fullsized":"2024-09-16T00:40:59.587479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588245+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:40:59.588245+0000","last_peered":"2024-09-16T00:40:59.588245+0000","last_clean":"2024-09-16T00:40:59.588245+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:40:59.588245+0000","last_undegraded":"2024-09-16T00:40:59.588245+0000","last_fullsized":"2024-09-16T00:40:59.588245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585828+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:40:59.585828+0000","last_peered":"2024-09-16T00:40:59.585828+0000","last_clean":"2024-09-16T00:40:59.585828+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:40:59.585828+0000","last_undegraded":"2024-09-16T00:40:59.585828+0000","last_fullsized":"2024-09-16T00:40:59.585828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595364+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:40:59.595364+0000","last_peered":"2024-09-16T00:40:59.595364+0000","last_clean":"2024-09-16T00:40:59.595364+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:40:59.595364+0000","last_undegraded":"2024-09-16T00:40:59.595364+0000","last_fullsized":"2024-09-16T00:40:59.595364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592178+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:40:59.592178+0000","last_peered":"2024-09-16T00:40:59.592178+0000","last_clean":"2024-09-16T00:40:59.592178+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:40:59.592178+0000","last_undegraded":"2024-09-16T00:40:59.592178+0000","last_fullsized":"2024-09-16T00:40:59.592178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590248+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:40:59.590248+0000","last_peered":"2024-09-16T00:40:59.590248+0000","last_clean":"2024-09-16T00:40:59.590248+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:40:59.590248+0000","last_undegraded":"2024-09-16T00:40:59.590248+0000","last_fullsized":"2024-09-16T00:40:59.590248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586021+0000","last_change":"2024-09-16T00:39:59.593568+0000","last_active":"2024-09-16T00:40:59.586021+0000","last_peered":"2024-09-16T00:40:59.586021+0000","last_clean":"2024-09-16T00:40:59.586021+0000","last_became_active":"2024-09-16T00:39:59.593209+0000","last_became_peered":"2024-09-16T00:39:59.593209+0000","last_unstale":"2024-09-16T00:40:59.586021+0000","last_undegraded":"2024-09-16T00:40:59.586021+0000","last_fullsized":"2024-09-16T00:40:59.586021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:56.516773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.596544+0000","last_change":"2024-09-16T00:40:59.596544+0000","last_active":"2024-09-16T00:40:59.596544+0000","last_peered":"2024-09-16T00:40:59.596544+0000","last_clean":"2024-09-16T00:40:59.596544+0000","last_became_active":"2024-09-16T00:40:59.596128+0000","last_became_peered":"2024-09-16T00:40:59.596128+0000","last_unstale":"2024-09-16T00:40:59.596544+0000","last_undegraded":"2024-09-16T00:40:59.596544+0000","last_fullsized":"2024-09-16T00:40:59.596544+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:22.691164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587645+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:40:59.587645+0000","last_peered":"2024-09-16T00:40:59.587645+0000","last_clean":"2024-09-16T00:40:59.587645+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:40:59.587645+0000","last_undegraded":"2024-09-16T00:40:59.587645+0000","last_fullsized":"2024-09-16T00:40:59.587645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747362+0000","last_change":"2024-09-16T00:39:59.380258+0000","last_active":"2024-09-16T00:40:59.747362+0000","last_peered":"2024-09-16T00:40:59.747362+0000","last_clean":"2024-09-16T00:40:59.747362+0000","last_became_active":"2024-09-16T00:39:59.379813+0000","last_became_peered":"2024-09-16T00:39:59.379813+0000","last_unstale":"2024-09-16T00:40:59.747362+0000","last_undegraded":"2024-09-16T00:40:59.747362+0000","last_fullsized":"2024-09-16T00:40:59.747362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:03.110305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595055+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:40:59.595055+0000","last_peered":"2024-09-16T00:40:59.595055+0000","last_clean":"2024-09-16T00:40:59.595055+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:40:59.595055+0000","last_undegraded":"2024-09-16T00:40:59.595055+0000","last_fullsized":"2024-09-16T00:40:59.595055+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588623+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:40:59.588623+0000","last_peered":"2024-09-16T00:40:59.588623+0000","last_clean":"2024-09-16T00:40:59.588623+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:40:59.588623+0000","last_undegraded":"2024-09-16T00:40:59.588623+0000","last_fullsized":"2024-09-16T00:40:59.588623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588982+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:40:59.588982+0000","last_peered":"2024-09-16T00:40:59.588982+0000","last_clean":"2024-09-16T00:40:59.588982+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:40:59.588982+0000","last_undegraded":"2024-09-16T00:40:59.588982+0000","last_fullsized":"2024-09-16T00:40:59.588982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586881+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:40:59.586881+0000","last_peered":"2024-09-16T00:40:59.586881+0000","last_clean":"2024-09-16T00:40:59.586881+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:40:59.586881+0000","last_undegraded":"2024-09-16T00:40:59.586881+0000","last_fullsized":"2024-09-16T00:40:59.586881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":265,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:05.289803+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:41:05.289803+0000","last_peered":"2024-09-16T00:41:05.289803+0000","last_clean":"2024-09-16T00:41:05.289803+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:41:05.289803+0000","last_undegraded":"2024-09-16T00:41:05.289803+0000","last_fullsized":"2024-09-16T00:41:05.289803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64068,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":193,"num_read_kb":159,"num_write":34,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585067+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:40:59.585067+0000","last_peered":"2024-09-16T00:40:59.585067+0000","last_clean":"2024-09-16T00:40:59.585067+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:40:59.585067+0000","last_undegraded":"2024-09-16T00:40:59.585067+0000","last_fullsized":"2024-09-16T00:40:59.585067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587571+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:40:59.587571+0000","last_peered":"2024-09-16T00:40:59.587571+0000","last_clean":"2024-09-16T00:40:59.587571+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:40:59.587571+0000","last_undegraded":"2024-09-16T00:40:59.587571+0000","last_fullsized":"2024-09-16T00:40:59.587571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589244+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:40:59.589244+0000","last_peered":"2024-09-16T00:40:59.589244+0000","last_clean":"2024-09-16T00:40:59.589244+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:40:59.589244+0000","last_undegraded":"2024-09-16T00:40:59.589244+0000","last_fullsized":"2024-09-16T00:40:59.589244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590297+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:40:59.590297+0000","last_peered":"2024-09-16T00:40:59.590297+0000","last_clean":"2024-09-16T00:40:59.590297+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:40:59.590297+0000","last_undegraded":"2024-09-16T00:40:59.590297+0000","last_fullsized":"2024-09-16T00:40:59.590297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162296+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:41:00.162296+0000","last_peered":"2024-09-16T00:41:00.162296+0000","last_clean":"2024-09-16T00:41:00.162296+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:41:00.162296+0000","last_undegraded":"2024-09-16T00:41:00.162296+0000","last_fullsized":"2024-09-16T00:41:00.162296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592831+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:40:59.592831+0000","last_peered":"2024-09-16T00:40:59.592831+0000","last_clean":"2024-09-16T00:40:59.592831+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:40:59.592831+0000","last_undegraded":"2024-09-16T00:40:59.592831+0000","last_fullsized":"2024-09-16T00:40:59.592831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584869+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:40:59.584869+0000","last_peered":"2024-09-16T00:40:59.584869+0000","last_clean":"2024-09-16T00:40:59.584869+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:40:59.584869+0000","last_undegraded":"2024-09-16T00:40:59.584869+0000","last_fullsized":"2024-09-16T00:40:59.584869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587926+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:40:59.587926+0000","last_peered":"2024-09-16T00:40:59.587926+0000","last_clean":"2024-09-16T00:40:59.587926+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:40:59.587926+0000","last_undegraded":"2024-09-16T00:40:59.587926+0000","last_fullsized":"2024-09-16T00:40:59.587926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747739+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:40:59.747739+0000","last_peered":"2024-09-16T00:40:59.747739+0000","last_clean":"2024-09-16T00:40:59.747739+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:40:59.747739+0000","last_undegraded":"2024-09-16T00:40:59.747739+0000","last_fullsized":"2024-09-16T00:40:59.747739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747139+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:40:59.747139+0000","last_peered":"2024-09-16T00:40:59.747139+0000","last_clean":"2024-09-16T00:40:59.747139+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:40:59.747139+0000","last_undegraded":"2024-09-16T00:40:59.747139+0000","last_fullsized":"2024-09-16T00:40:59.747139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161443+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:41:00.161443+0000","last_peered":"2024-09-16T00:41:00.161443+0000","last_clean":"2024-09-16T00:41:00.161443+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:41:00.161443+0000","last_undegraded":"2024-09-16T00:41:00.161443+0000","last_fullsized":"2024-09-16T00:41:00.161443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584969+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:40:59.584969+0000","last_peered":"2024-09-16T00:40:59.584969+0000","last_clean":"2024-09-16T00:40:59.584969+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:40:59.584969+0000","last_undegraded":"2024-09-16T00:40:59.584969+0000","last_fullsized":"2024-09-16T00:40:59.584969+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588578+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:40:59.588578+0000","last_peered":"2024-09-16T00:40:59.588578+0000","last_clean":"2024-09-16T00:40:59.588578+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:40:59.588578+0000","last_undegraded":"2024-09-16T00:40:59.588578+0000","last_fullsized":"2024-09-16T00:40:59.588578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747585+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:40:59.747585+0000","last_peered":"2024-09-16T00:40:59.747585+0000","last_clean":"2024-09-16T00:40:59.747585+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:40:59.747585+0000","last_undegraded":"2024-09-16T00:40:59.747585+0000","last_fullsized":"2024-09-16T00:40:59.747585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747194+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:40:59.747194+0000","last_peered":"2024-09-16T00:40:59.747194+0000","last_clean":"2024-09-16T00:40:59.747194+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:40:59.747194+0000","last_undegraded":"2024-09-16T00:40:59.747194+0000","last_fullsized":"2024-09-16T00:40:59.747194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594858+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:40:59.594858+0000","last_peered":"2024-09-16T00:40:59.594858+0000","last_clean":"2024-09-16T00:40:59.594858+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:40:59.594858+0000","last_undegraded":"2024-09-16T00:40:59.594858+0000","last_fullsized":"2024-09-16T00:40:59.594858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747376+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:40:59.747376+0000","last_peered":"2024-09-16T00:40:59.747376+0000","last_clean":"2024-09-16T00:40:59.747376+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:40:59.747376+0000","last_undegraded":"2024-09-16T00:40:59.747376+0000","last_fullsized":"2024-09-16T00:40:59.747376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588250+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:40:59.588250+0000","last_peered":"2024-09-16T00:40:59.588250+0000","last_clean":"2024-09-16T00:40:59.588250+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:40:59.588250+0000","last_undegraded":"2024-09-16T00:40:59.588250+0000","last_fullsized":"2024-09-16T00:40:59.588250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587684+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:40:59.587684+0000","last_peered":"2024-09-16T00:40:59.587684+0000","last_clean":"2024-09-16T00:40:59.587684+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:40:59.587684+0000","last_undegraded":"2024-09-16T00:40:59.587684+0000","last_fullsized":"2024-09-16T00:40:59.587684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588368+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:40:59.588368+0000","last_peered":"2024-09-16T00:40:59.588368+0000","last_clean":"2024-09-16T00:40:59.588368+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:40:59.588368+0000","last_undegraded":"2024-09-16T00:40:59.588368+0000","last_fullsized":"2024-09-16T00:40:59.588368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586131+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:40:59.586131+0000","last_peered":"2024-09-16T00:40:59.586131+0000","last_clean":"2024-09-16T00:40:59.586131+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:40:59.586131+0000","last_undegraded":"2024-09-16T00:40:59.586131+0000","last_fullsized":"2024-09-16T00:40:59.586131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588144+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:40:59.588144+0000","last_peered":"2024-09-16T00:40:59.588144+0000","last_clean":"2024-09-16T00:40:59.588144+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:40:59.588144+0000","last_undegraded":"2024-09-16T00:40:59.588144+0000","last_fullsized":"2024-09-16T00:40:59.588144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747412+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:40:59.747412+0000","last_peered":"2024-09-16T00:40:59.747412+0000","last_clean":"2024-09-16T00:40:59.747412+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:40:59.747412+0000","last_undegraded":"2024-09-16T00:40:59.747412+0000","last_fullsized":"2024-09-16T00:40:59.747412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588676+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:40:59.588676+0000","last_peered":"2024-09-16T00:40:59.588676+0000","last_clean":"2024-09-16T00:40:59.588676+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:40:59.588676+0000","last_undegraded":"2024-09-16T00:40:59.588676+0000","last_fullsized":"2024-09-16T00:40:59.588676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587600+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:40:59.587600+0000","last_peered":"2024-09-16T00:40:59.587600+0000","last_clean":"2024-09-16T00:40:59.587600+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:40:59.587600+0000","last_undegraded":"2024-09-16T00:40:59.587600+0000","last_fullsized":"2024-09-16T00:40:59.587600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162004+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:41:00.162004+0000","last_peered":"2024-09-16T00:41:00.162004+0000","last_clean":"2024-09-16T00:41:00.162004+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:41:00.162004+0000","last_undegraded":"2024-09-16T00:41:00.162004+0000","last_fullsized":"2024-09-16T00:41:00.162004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595476+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:40:59.595476+0000","last_peered":"2024-09-16T00:40:59.595476+0000","last_clean":"2024-09-16T00:40:59.595476+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:40:59.595476+0000","last_undegraded":"2024-09-16T00:40:59.595476+0000","last_fullsized":"2024-09-16T00:40:59.595476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747786+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:40:59.747786+0000","last_peered":"2024-09-16T00:40:59.747786+0000","last_clean":"2024-09-16T00:40:59.747786+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:40:59.747786+0000","last_undegraded":"2024-09-16T00:40:59.747786+0000","last_fullsized":"2024-09-16T00:40:59.747786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588960+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:40:59.588960+0000","last_peered":"2024-09-16T00:40:59.588960+0000","last_clean":"2024-09-16T00:40:59.588960+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:40:59.588960+0000","last_undegraded":"2024-09-16T00:40:59.588960+0000","last_fullsized":"2024-09-16T00:40:59.588960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.748038+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:40:59.748038+0000","last_peered":"2024-09-16T00:40:59.748038+0000","last_clean":"2024-09-16T00:40:59.748038+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:40:59.748038+0000","last_undegraded":"2024-09-16T00:40:59.748038+0000","last_fullsized":"2024-09-16T00:40:59.748038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587358+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:40:59.587358+0000","last_peered":"2024-09-16T00:40:59.587358+0000","last_clean":"2024-09-16T00:40:59.587358+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:40:59.587358+0000","last_undegraded":"2024-09-16T00:40:59.587358+0000","last_fullsized":"2024-09-16T00:40:59.587358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747240+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:40:59.747240+0000","last_peered":"2024-09-16T00:40:59.747240+0000","last_clean":"2024-09-16T00:40:59.747240+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:40:59.747240+0000","last_undegraded":"2024-09-16T00:40:59.747240+0000","last_fullsized":"2024-09-16T00:40:59.747240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587485+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:40:59.587485+0000","last_peered":"2024-09-16T00:40:59.587485+0000","last_clean":"2024-09-16T00:40:59.587485+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:40:59.587485+0000","last_undegraded":"2024-09-16T00:40:59.587485+0000","last_fullsized":"2024-09-16T00:40:59.587485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595614+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:40:59.595614+0000","last_peered":"2024-09-16T00:40:59.595614+0000","last_clean":"2024-09-16T00:40:59.595614+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:40:59.595614+0000","last_undegraded":"2024-09-16T00:40:59.595614+0000","last_fullsized":"2024-09-16T00:40:59.595614+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747469+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:40:59.747469+0000","last_peered":"2024-09-16T00:40:59.747469+0000","last_clean":"2024-09-16T00:40:59.747469+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:40:59.747469+0000","last_undegraded":"2024-09-16T00:40:59.747469+0000","last_fullsized":"2024-09-16T00:40:59.747469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588263+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:40:59.588263+0000","last_peered":"2024-09-16T00:40:59.588263+0000","last_clean":"2024-09-16T00:40:59.588263+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:40:59.588263+0000","last_undegraded":"2024-09-16T00:40:59.588263+0000","last_fullsized":"2024-09-16T00:40:59.588263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586045+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:40:59.586045+0000","last_peered":"2024-09-16T00:40:59.586045+0000","last_clean":"2024-09-16T00:40:59.586045+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:40:59.586045+0000","last_undegraded":"2024-09-16T00:40:59.586045+0000","last_fullsized":"2024-09-16T00:40:59.586045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747774+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:40:59.747774+0000","last_peered":"2024-09-16T00:40:59.747774+0000","last_clean":"2024-09-16T00:40:59.747774+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:40:59.747774+0000","last_undegraded":"2024-09-16T00:40:59.747774+0000","last_fullsized":"2024-09-16T00:40:59.747774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587142+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:40:59.587142+0000","last_peered":"2024-09-16T00:40:59.587142+0000","last_clean":"2024-09-16T00:40:59.587142+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:40:59.587142+0000","last_undegraded":"2024-09-16T00:40:59.587142+0000","last_fullsized":"2024-09-16T00:40:59.587142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161639+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:41:00.161639+0000","last_peered":"2024-09-16T00:41:00.161639+0000","last_clean":"2024-09-16T00:41:00.161639+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:41:00.161639+0000","last_undegraded":"2024-09-16T00:41:00.161639+0000","last_fullsized":"2024-09-16T00:41:00.161639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161754+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:41:00.161754+0000","last_peered":"2024-09-16T00:41:00.161754+0000","last_clean":"2024-09-16T00:41:00.161754+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:41:00.161754+0000","last_undegraded":"2024-09-16T00:41:00.161754+0000","last_fullsized":"2024-09-16T00:41:00.161754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595267+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:40:59.595267+0000","last_peered":"2024-09-16T00:40:59.595267+0000","last_clean":"2024-09-16T00:40:59.595267+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:40:59.595267+0000","last_undegraded":"2024-09-16T00:40:59.595267+0000","last_fullsized":"2024-09-16T00:40:59.595267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590469+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:40:59.590469+0000","last_peered":"2024-09-16T00:40:59.590469+0000","last_clean":"2024-09-16T00:40:59.590469+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:40:59.590469+0000","last_undegraded":"2024-09-16T00:40:59.590469+0000","last_fullsized":"2024-09-16T00:40:59.590469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592840+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:40:59.592840+0000","last_peered":"2024-09-16T00:40:59.592840+0000","last_clean":"2024-09-16T00:40:59.592840+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:40:59.592840+0000","last_undegraded":"2024-09-16T00:40:59.592840+0000","last_fullsized":"2024-09-16T00:40:59.592840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747793+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:40:59.747793+0000","last_peered":"2024-09-16T00:40:59.747793+0000","last_clean":"2024-09-16T00:40:59.747793+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:40:59.747793+0000","last_undegraded":"2024-09-16T00:40:59.747793+0000","last_fullsized":"2024-09-16T00:40:59.747793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587933+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:40:59.587933+0000","last_peered":"2024-09-16T00:40:59.587933+0000","last_clean":"2024-09-16T00:40:59.587933+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:40:59.587933+0000","last_undegraded":"2024-09-16T00:40:59.587933+0000","last_fullsized":"2024-09-16T00:40:59.587933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747138+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:40:59.747138+0000","last_peered":"2024-09-16T00:40:59.747138+0000","last_clean":"2024-09-16T00:40:59.747138+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:40:59.747138+0000","last_undegraded":"2024-09-16T00:40:59.747138+0000","last_fullsized":"2024-09-16T00:40:59.747138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592328+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:40:59.592328+0000","last_peered":"2024-09-16T00:40:59.592328+0000","last_clean":"2024-09-16T00:40:59.592328+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:40:59.592328+0000","last_undegraded":"2024-09-16T00:40:59.592328+0000","last_fullsized":"2024-09-16T00:40:59.592328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747364+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:40:59.747364+0000","last_peered":"2024-09-16T00:40:59.747364+0000","last_clean":"2024-09-16T00:40:59.747364+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:40:59.747364+0000","last_undegraded":"2024-09-16T00:40:59.747364+0000","last_fullsized":"2024-09-16T00:40:59.747364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586035+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:40:59.586035+0000","last_peered":"2024-09-16T00:40:59.586035+0000","last_clean":"2024-09-16T00:40:59.586035+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:40:59.586035+0000","last_undegraded":"2024-09-16T00:40:59.586035+0000","last_fullsized":"2024-09-16T00:40:59.586035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590395+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:40:59.590395+0000","last_peered":"2024-09-16T00:40:59.590395+0000","last_clean":"2024-09-16T00:40:59.590395+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:40:59.590395+0000","last_undegraded":"2024-09-16T00:40:59.590395+0000","last_fullsized":"2024-09-16T00:40:59.590395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162106+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:41:00.162106+0000","last_peered":"2024-09-16T00:41:00.162106+0000","last_clean":"2024-09-16T00:41:00.162106+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:41:00.162106+0000","last_undegraded":"2024-09-16T00:41:00.162106+0000","last_fullsized":"2024-09-16T00:41:00.162106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162139+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:41:00.162139+0000","last_peered":"2024-09-16T00:41:00.162139+0000","last_clean":"2024-09-16T00:41:00.162139+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:41:00.162139+0000","last_undegraded":"2024-09-16T00:41:00.162139+0000","last_fullsized":"2024-09-16T00:41:00.162139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747779+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:40:59.747779+0000","last_peered":"2024-09-16T00:40:59.747779+0000","last_clean":"2024-09-16T00:40:59.747779+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:40:59.747779+0000","last_undegraded":"2024-09-16T00:40:59.747779+0000","last_fullsized":"2024-09-16T00:40:59.747779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64866,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":199,"num_read_kb":165,"num_write":91,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":194598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547241,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":518379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776174,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37028,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708124,"statfs":{"total":95995035648,"available":95957118976,"internally_reserved":0,"allocated":1716224,"data_stored":1173009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939698,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":518961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168632,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":518983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397565,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1638400,"data_stored":1113425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":514283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1642496,"data_stored":1108929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":518401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215185,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":514387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444119,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1110016,"data_stored":578385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1105920,"data_stored":578351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":514373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:41:08.821 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:09.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:08 smithi102 ceph-mon[35364]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 449 B/s wr, 3 op/s 2024-09-16T00:41:09.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:08 smithi102 ceph-mon[35364]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:09.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:08 smithi102 ceph-mon[35364]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:09.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:08 smithi031 ceph-mon[30956]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 449 B/s wr, 3 op/s 2024-09-16T00:41:09.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:08 smithi031 ceph-mon[30956]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:09.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:08 smithi031 ceph-mon[30956]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:09.128 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:08 smithi081 ceph-mon[35539]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 449 B/s wr, 3 op/s 2024-09-16T00:41:09.128 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:08 smithi081 ceph-mon[35539]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:09.128 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:08 smithi081 ceph-mon[35539]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 -- 172.21.15.31:0/3253758141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 msgr2=0x7fb64010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 --2- 172.21.15.31:0/3253758141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 0x7fb64010dfb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb630009900 tx=0x7fb63002a410 comp rx=0 tx=0).stop 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 -- 172.21.15.31:0/3253758141 shutdown_connections 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 --2- 172.21.15.31:0/3253758141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb64010e4f0 0x7fb640110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 --2- 172.21.15.31:0/3253758141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 0x7fb64010dfb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 --2- 172.21.15.31:0/3253758141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6401009d0 0x7fb640100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 -- 172.21.15.31:0/3253758141 >> 172.21.15.31:0/3253758141 conn(0x7fb6400fc7d0 msgr2=0x7fb6400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 -- 172.21.15.31:0/3253758141 shutdown_connections 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 -- 172.21.15.31:0/3253758141 wait complete. 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 Processor -- start 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.607+0000 7fb6474c6640 1 -- start start 2024-09-16T00:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb6474c6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6401009d0 0x7fb6401a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb6474c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 0x7fb6401a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb6474c6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb64010e4f0 0x7fb64019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb6474c6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6401142c0 con 0x7fb640101380 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb6474c6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb640114140 con 0x7fb6401009d0 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb6474c6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb640114440 con 0x7fb64010e4f0 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb644a61640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 0x7fb6401a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb644a61640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 0x7fb6401a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60346/0 (socket says 172.21.15.31:60346) 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb644a61640 1 -- 172.21.15.31:0/4072794750 learned_addr learned my addr 172.21.15.31:0/4072794750 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb645262640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6401009d0 0x7fb6401a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb645a63640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb64010e4f0 0x7fb64019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb644a61640 1 -- 172.21.15.31:0/4072794750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb64010e4f0 msgr2=0x7fb64019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb644a61640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb64010e4f0 0x7fb64019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb644a61640 1 -- 172.21.15.31:0/4072794750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6401009d0 msgr2=0x7fb6401a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb644a61640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6401009d0 0x7fb6401a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb644a61640 1 -- 172.21.15.31:0/4072794750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64019ad10 con 0x7fb640101380 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb644a61640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 0x7fb6401a0960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb630006010 tx=0x7fb6300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.608+0000 7fb6367fc640 1 -- 172.21.15.31:0/4072794750 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63003f030 con 0x7fb640101380 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.609+0000 7fb6367fc640 1 -- 172.21.15.31:0/4072794750 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb630003ce0 con 0x7fb640101380 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.609+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64019afa0 con 0x7fb640101380 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.609+0000 7fb6367fc640 1 -- 172.21.15.31:0/4072794750 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630035070 con 0x7fb640101380 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.609+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6401a72f0 con 0x7fb640101380 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.610+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb640100db0 con 0x7fb640101380 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.612+0000 7fb6367fc640 1 -- 172.21.15.31:0/4072794750 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb630042030 con 0x7fb640101380 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.612+0000 7fb6367fc640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb62c0a32c0 0x7fb62c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.613+0000 7fb645262640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb62c0a32c0 0x7fb62c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.613+0000 7fb6367fc640 1 -- 172.21.15.31:0/4072794750 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb6300f05e0 con 0x7fb640101380 2024-09-16T00:41:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.613+0000 7fb645262640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb62c0a32c0 0x7fb62c0a5780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005fd0 tx=0x7fb63c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:09.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.616+0000 7fb6367fc640 1 -- 172.21.15.31:0/4072794750 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6300bd010 con 0x7fb640101380 2024-09-16T00:41:09.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.827+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb64019bb10 con 0x7fb640101380 2024-09-16T00:41:09.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:09 smithi031 ceph-mon[30956]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:09.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.828+0000 7fb6367fc640 1 -- 172.21.15.31:0/4072794750 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fb64019bb10 con 0x7fb640101380 2024-09-16T00:41:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:09.829 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:41:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:41:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb62c0a32c0 msgr2=0x7fb62c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb62c0a32c0 0x7fb62c0a5780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005fd0 tx=0x7fb63c009680 comp rx=0 tx=0).stop 2024-09-16T00:41:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 msgr2=0x7fb6401a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 0x7fb6401a0960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb630006010 tx=0x7fb6300042f0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 shutdown_connections 2024-09-16T00:41:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb62c0a32c0 0x7fb62c0a5780 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb64010e4f0 0x7fb64019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb640101380 0x7fb6401a0960 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 --2- 172.21.15.31:0/4072794750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6401009d0 0x7fb6401a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 >> 172.21.15.31:0/4072794750 conn(0x7fb6400fc7d0 msgr2=0x7fb6400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 shutdown_connections 2024-09-16T00:41:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:09.830+0000 7fb6474c6640 1 -- 172.21.15.31:0/4072794750 wait complete. 2024-09-16T00:41:10.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:09 smithi102 ceph-mon[35364]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:10.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:09 smithi081 ceph-mon[35539]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:10.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:41:10.944 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:11.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:10 smithi102 ceph-mon[35364]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 506 B/s wr, 2 op/s 2024-09-16T00:41:11.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:10 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4072794750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:11.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:10 smithi031 ceph-mon[30956]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 506 B/s wr, 2 op/s 2024-09-16T00:41:11.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:10 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4072794750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:11.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:10 smithi081 ceph-mon[35539]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 506 B/s wr, 2 op/s 2024-09-16T00:41:11.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:10 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4072794750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.464+0000 7f6270f0c640 1 -- 172.21.15.31:0/1125418492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c100ca0 msgr2=0x7f626c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.464+0000 7f6270f0c640 1 --2- 172.21.15.31:0/1125418492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c100ca0 0x7f626c106840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6254009930 tx=0x7f625402a420 comp rx=0 tx=0).stop 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.464+0000 7f6270f0c640 1 -- 172.21.15.31:0/1125418492 shutdown_connections 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.464+0000 7f6270f0c640 1 --2- 172.21.15.31:0/1125418492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626c106f50 0x7f626c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.464+0000 7f6270f0c640 1 --2- 172.21.15.31:0/1125418492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c100ca0 0x7f626c106840 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.464+0000 7f6270f0c640 1 --2- 172.21.15.31:0/1125418492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f626c1002f0 0x7f626c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.464+0000 7f6270f0c640 1 -- 172.21.15.31:0/1125418492 >> 172.21.15.31:0/1125418492 conn(0x7f626c077210 msgr2=0x7f626c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.464+0000 7f6270f0c640 1 -- 172.21.15.31:0/1125418492 shutdown_connections 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.465+0000 7f6270f0c640 1 -- 172.21.15.31:0/1125418492 wait complete. 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.465+0000 7f6270f0c640 1 Processor -- start 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.465+0000 7f6270f0c640 1 -- start start 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.465+0000 7f6270f0c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c1002f0 0x7f626c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f6270f0c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626c100ca0 0x7f626c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f6270f0c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f626c106f50 0x7f626c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f6270f0c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f626c111fa0 con 0x7f626c1002f0 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f6270f0c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f626c111e20 con 0x7f626c100ca0 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f6270f0c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f626c112120 con 0x7f626c106f50 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c1002f0 0x7f626c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c1002f0 0x7f626c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60362/0 (socket says 172.21.15.31:60362) 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626a59c640 1 -- 172.21.15.31:0/4262953606 learned_addr learned my addr 172.21.15.31:0/4262953606 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f6263fff640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626c100ca0 0x7f626c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626ad9d640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f626c106f50 0x7f626c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626a59c640 1 -- 172.21.15.31:0/4262953606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f626c106f50 msgr2=0x7f626c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626a59c640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f626c106f50 0x7f626c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626a59c640 1 -- 172.21.15.31:0/4262953606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626c100ca0 msgr2=0x7f626c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626a59c640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626c100ca0 0x7f626c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626a59c640 1 -- 172.21.15.31:0/4262953606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c19d4d0 con 0x7f626c1002f0 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f626a59c640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c1002f0 0x7f626c198520 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f625c00d770 tx=0x7f625c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f62637fe640 1 -- 172.21.15.31:0/4262953606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c017070 con 0x7f626c1002f0 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f62637fe640 1 -- 172.21.15.31:0/4262953606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c005500 con 0x7f626c1002f0 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c19d7c0 con 0x7f626c1002f0 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f62637fe640 1 -- 172.21.15.31:0/4262953606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c004e00 con 0x7f626c1002f0 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.466+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c1a50a0 con 0x7f626c1002f0 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.468+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626c1006d0 con 0x7f626c1002f0 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.469+0000 7f62637fe640 1 -- 172.21.15.31:0/4262953606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f625c004fa0 con 0x7f626c1002f0 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.469+0000 7f62637fe640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62580a3240 0x7f62580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.470+0000 7f6263fff640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62580a3240 0x7f62580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.470+0000 7f62637fe640 1 -- 172.21.15.31:0/4262953606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f625c0cfc80 con 0x7f626c1002f0 2024-09-16T00:41:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.470+0000 7f6263fff640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62580a3240 0x7f62580a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6254006010 tx=0x7f62540314b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.473+0000 7f62637fe640 1 -- 172.21.15.31:0/4262953606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f625c09c590 con 0x7f626c1002f0 2024-09-16T00:41:11.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.649+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f626c10a120 con 0x7f626c1002f0 2024-09-16T00:41:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.651+0000 7f62637fe640 1 -- 172.21.15.31:0/4262953606 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24766 (secure 0 0 0) 0x7f625c09be90 con 0x7f626c1002f0 2024-09-16T00:41:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:11.652 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:40:59.575845+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62580a3240 msgr2=0x7f62580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62580a3240 0x7f62580a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6254006010 tx=0x7f62540314b0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c1002f0 msgr2=0x7f626c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c1002f0 0x7f626c198520 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f625c00d770 tx=0x7f625c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 shutdown_connections 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62580a3240 0x7f62580a5700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f626c106f50 0x7f626c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f626c100ca0 0x7f626c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 --2- 172.21.15.31:0/4262953606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f626c1002f0 0x7f626c198520 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 >> 172.21.15.31:0/4262953606 conn(0x7f626c077210 msgr2=0x7f626c105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 shutdown_connections 2024-09-16T00:41:11.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:11.653+0000 7f6270f0c640 1 -- 172.21.15.31:0/4262953606 wait complete. 2024-09-16T00:41:11.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:11 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4262953606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:12.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:11 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4262953606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:12.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:11 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4262953606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:12.318 DEBUG:tasks.fwd_scrub:Yielding 2024-09-16T00:41:12.318 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-16T00:41:12.330 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:41:12.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:41:12.331 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-16T00:41:12.331 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:41:12.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:41:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:12.609 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:12.998 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:12 smithi031 ceph-mon[30956]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 938 B/s wr, 3 op/s 2024-09-16T00:41:13.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:12 smithi102 ceph-mon[35364]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 938 B/s wr, 3 op/s 2024-09-16T00:41:13.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:12 smithi081 ceph-mon[35539]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 938 B/s wr, 3 op/s 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.322+0000 7f58dbbde640 1 -- 172.21.15.31:0/1594746668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074620 msgr2=0x7f58d4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.322+0000 7f58dbbde640 1 --2- 172.21.15.31:0/1594746668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074620 0x7f58d4074a00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009ca0 tx=0x7f58c402a950 comp rx=0 tx=0).stop 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.322+0000 7f58dbbde640 1 -- 172.21.15.31:0/1594746668 shutdown_connections 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.322+0000 7f58dbbde640 1 --2- 172.21.15.31:0/1594746668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d410e3c0 0x7f58d4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.322+0000 7f58dbbde640 1 --2- 172.21.15.31:0/1594746668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58d4074f40 0x7f58d410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.322+0000 7f58dbbde640 1 --2- 172.21.15.31:0/1594746668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074620 0x7f58d4074a00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.322+0000 7f58dbbde640 1 -- 172.21.15.31:0/1594746668 >> 172.21.15.31:0/1594746668 conn(0x7f58d40fd8c0 msgr2=0x7f58d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.322+0000 7f58dbbde640 1 -- 172.21.15.31:0/1594746668 shutdown_connections 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.322+0000 7f58dbbde640 1 -- 172.21.15.31:0/1594746668 wait complete. 2024-09-16T00:41:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58dbbde640 1 Processor -- start 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58dbbde640 1 -- start start 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58dbbde640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d4074620 0x7f58d419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58dbbde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074f40 0x7f58d419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58dbbde640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58d410e3c0 0x7f58d419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58dbbde640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4114150 con 0x7f58d4074f40 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58dbbde640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4113fd0 con 0x7f58d4074620 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58dbbde640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41142d0 con 0x7f58d410e3c0 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58d9179640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074f40 0x7f58d419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58d9179640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074f40 0x7f58d419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60374/0 (socket says 172.21.15.31:60374) 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58d9179640 1 -- 172.21.15.31:0/3191200187 learned_addr learned my addr 172.21.15.31:0/3191200187 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58d997a640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d4074620 0x7f58d419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58d9179640 1 -- 172.21.15.31:0/3191200187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58d410e3c0 msgr2=0x7f58d419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58da17b640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58d410e3c0 0x7f58d419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58d9179640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58d410e3c0 0x7f58d419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58d9179640 1 -- 172.21.15.31:0/3191200187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d4074620 msgr2=0x7f58d419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58d9179640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d4074620 0x7f58d419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58d9179640 1 -- 172.21.15.31:0/3191200187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d419f600 con 0x7f58d4074f40 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.323+0000 7f58da17b640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58d410e3c0 0x7f58d419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.324+0000 7f58d9179640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074f40 0x7f58d419ab90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c890 tx=0x7f58c800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.324+0000 7f58d2ffd640 1 -- 172.21.15.31:0/3191200187 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8004220 con 0x7f58d4074f40 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.324+0000 7f58dbbde640 1 -- 172.21.15.31:0/3191200187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d419f8f0 con 0x7f58d4074f40 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.324+0000 7f58d2ffd640 1 -- 172.21.15.31:0/3191200187 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c80043c0 con 0x7f58d4074f40 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.324+0000 7f58d2ffd640 1 -- 172.21.15.31:0/3191200187 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8004a00 con 0x7f58d4074f40 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.324+0000 7f58dbbde640 1 -- 172.21.15.31:0/3191200187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d41a71d0 con 0x7f58d4074f40 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.326+0000 7f58d2ffd640 1 -- 172.21.15.31:0/3191200187 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f58c80079b0 con 0x7f58d4074f40 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58ddfb7740 1 Processor -- start 2024-09-16T00:41:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58ddfb7740 1 -- start start 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a16f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a21800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 0x55a8c2a24130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a8c297c280 con 0x55a8c2a16b60 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2923b10 con 0x55a8c2a17480 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2978240 con 0x55a8c2a21d40 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58da17b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a16f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58da17b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a16f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60382/0 (socket says 172.21.15.31:60382) 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58da17b640 1 -- 172.21.15.31:0/261206535 learned_addr learned my addr 172.21.15.31:0/261206535 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58d9179640 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 0x55a8c2a24130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.327+0000 7f58d997a640 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a21800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58d2ffd640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58bc0a3240 0x7f58bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58da17b640 1 -- 172.21.15.31:0/261206535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 msgr2=0x55a8c2a24130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58da17b640 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 0x55a8c2a24130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58da17b640 1 -- 172.21.15.31:0/261206535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 msgr2=0x55a8c2a21800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58da17b640 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a21800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58da17b640 1 -- 172.21.15.31:0/261206535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c29301f0 con 0x55a8c2a16b60 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58d997a640 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a21800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58d2ffd640 1 -- 172.21.15.31:0/3191200187 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f58c80cf9c0 con 0x7f58d4074f40 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58da17b640 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a16f40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009b50 tx=0x7f58cc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619be0857af4e375 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58da17b640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58bc0a3240 0x7f58bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58bbfff640 1 -- 172.21.15.31:0/261206535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc01e6d0 con 0x55a8c2a16b60 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58bbfff640 1 -- 172.21.15.31:0/261206535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58cc016070 con 0x55a8c2a16b60 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.328+0000 7f58bbfff640 1 -- 172.21.15.31:0/261206535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc01dcb0 con 0x55a8c2a16b60 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 -- 172.21.15.31:0/261206535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 msgr2=0x55a8c2a16f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a16f40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009b50 tx=0x7f58cc00e6c0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58da17b640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58bc0a3240 0x7f58bc0a5700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f58cc01d000 tx=0x7f58cc020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 -- 172.21.15.31:0/261206535 shutdown_connections 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 0x55a8c2a24130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a21800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/261206535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a16f40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 -- 172.21.15.31:0/261206535 >> 172.21.15.31:0/261206535 conn(0x55a8c2a130f0 msgr2=0x55a8c2a134d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 -- 172.21.15.31:0/261206535 shutdown_connections 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 -- 172.21.15.31:0/261206535 wait complete. 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.329+0000 7f58ddfb7740 1 -- start start 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58ddfb7740 1 Processor -- start 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a1ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a1f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2a24910 con 0x55a8c2a16b60 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2936f10 con 0x55a8c2a17480 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2a24a90 con 0x55a8c2a21d40 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58ddfb7740 10 client.0 _ll_register_callbacks cb 0x55a8c28ffa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d997a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a1ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d997a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a1ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60394/0 (socket says 172.21.15.31:60394) 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d997a640 1 -- 172.21.15.31:0/2686609320 learned_addr learned my addr 172.21.15.31:0/2686609320 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d9179640 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a1f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d997a640 1 -- 172.21.15.31:0/2686609320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 msgr2=0x55a8c2a1f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58da17b640 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d997a640 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d997a640 1 -- 172.21.15.31:0/2686609320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 msgr2=0x55a8c2a1f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d997a640 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a1f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d997a640 1 -- 172.21.15.31:0/2686609320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c26ed6a0 con 0x55a8c2a16b60 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.330+0000 7f58d9179640 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a1f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58d997a640 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a1ec50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f58c4035000 tx=0x7f58c402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7f64914b49e885 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58b97fa640 1 -- 172.21.15.31:0/2686609320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4049060 con 0x55a8c2a16b60 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2686609320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58cc016070 con 0x55a8c2a16b60 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58b97fa640 1 -- 172.21.15.31:0/2686609320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f58c4003970 con 0x55a8c2a16b60 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58b97fa640 1 -- 172.21.15.31:0/2686609320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f58c4050050 con 0x55a8c2a16b60 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58ddfb7740 10 client.15213 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2686609320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a8c2a30540 con 0x55a8c2a16b60 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58b97fa640 1 -- 172.21.15.31:0/2686609320 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58cc016070 con 0x55a8c2a16b60 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58b97fa640 1 -- 172.21.15.31:0/2686609320 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f58c40304d0 con 0x55a8c2a16b60 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58ddfb7740 10 client.15213 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58ddfb7740 10 client.15213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58ddfb7740 20 client.15213 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a8c2a31a00 0x55a8c2a33e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58ddfb7740 4 client.15213 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.331+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2686609320 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8c29897f0 con 0x55a8c2a31a00 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.332+0000 7f58d9179640 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a8c2a31a00 0x55a8c2a33e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.332+0000 7f58d9179640 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a8c2a31a00 0x55a8c2a33e70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d915f9347ee355d7 server_cookie=8d3d2c74a170d398 in_seq=0 out_seq=0 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.332+0000 7f58b97fa640 10 client.15213 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58b97fa640 1 -- 172.21.15.31:0/2686609320 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8c29897f0 con 0x55a8c2a31a00 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58b97fa640 10 client.15213 handle_command_reply: tid=0 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 client.15213 shutdown 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2686609320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 msgr2=0x55a8c2a1ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a1ec50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f58c4035000 tx=0x7f58c402f3a0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2686609320 shutdown_connections 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a8c2a31a00 0x55a8c2a33e70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a17480 0x55a8c2a1f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2686609320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a1ec50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58b97fa640 0 client.15213 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2686609320 >> 172.21.15.31:0/2686609320 conn(0x55a8c2a130f0 msgr2=0x55a8c2a12270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2686609320 shutdown_connections 2024-09-16T00:41:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2686609320 wait complete. 2024-09-16T00:41:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.333+0000 7f58ddfb7740 20 client.15213 trim_cache size 0 max 16384 2024-09-16T00:41:13.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.351+0000 7f58ddfb7740 1 Processor -- start 2024-09-16T00:41:13.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.351+0000 7f58ddfb7740 1 -- start start 2024-09-16T00:41:13.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a16b60 0x55a8c2a396f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 0x55a8c2a39c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41af000 con 0x55a8c2a21d40 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41b1880 con 0x55a8c2a16b60 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41af670 con 0x55a8c2a17480 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58da17b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58da17b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60402/0 (socket says 172.21.15.31:60402) 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58d997a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a16b60 0x55a8c2a396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58da17b640 1 -- 172.21.15.31:0/2631919648 learned_addr learned my addr 172.21.15.31:0/2631919648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58d9179640 1 --2- 172.21.15.31:0/2631919648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 0x55a8c2a39c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58da17b640 1 -- 172.21.15.31:0/2631919648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 msgr2=0x55a8c2a39c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58da17b640 1 --2- 172.21.15.31:0/2631919648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 0x55a8c2a39c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58da17b640 1 -- 172.21.15.31:0/2631919648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a16b60 msgr2=0x55a8c2a396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58da17b640 1 --2- 172.21.15.31:0/2631919648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a16b60 0x55a8c2a396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58da17b640 1 -- 172.21.15.31:0/2631919648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c2a12b90 con 0x55a8c2a21d40 2024-09-16T00:41:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.352+0000 7f58da17b640 1 --2- 172.21.15.31:0/2631919648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1beb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004700 tx=0x7f58cc023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378a8d80f476a68f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:13.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58bb7fe640 1 -- 172.21.15.31:0/2631919648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc01e670 con 0x55a8c2a21d40 2024-09-16T00:41:13.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58bb7fe640 1 -- 172.21.15.31:0/2631919648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58cc02b070 con 0x55a8c2a21d40 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58bb7fe640 1 -- 172.21.15.31:0/2631919648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc01d890 con 0x55a8c2a21d40 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2631919648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a21d40 msgr2=0x55a8c2a1beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2631919648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1beb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004700 tx=0x7f58cc023000 comp rx=0 tx=0).stop 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2631919648 shutdown_connections 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2631919648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a21d40 0x55a8c2a1beb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2631919648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 0x55a8c2a39c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/2631919648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a16b60 0x55a8c2a396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2631919648 >> 172.21.15.31:0/2631919648 conn(0x55a8c2a130f0 msgr2=0x55a8c2a20bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2631919648 shutdown_connections 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.353+0000 7f58ddfb7740 1 -- 172.21.15.31:0/2631919648 wait complete. 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58ddfb7740 1 -- start start 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58ddfb7740 1 Processor -- start 2024-09-16T00:41:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a2a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 0x55a8c2a2a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58ddfb7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a21d40 0x55a8c2a26620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41afd40 con 0x55a8c2a16b60 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58d997a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a2a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41b11b0 con 0x55a8c2a21d40 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58ddfb7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41b0410 con 0x55a8c2a17480 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58d997a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a2a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60408/0 (socket says 172.21.15.31:60408) 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58d997a640 1 -- 172.21.15.31:0/610741361 learned_addr learned my addr 172.21.15.31:0/610741361 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.354+0000 7f58ddfb7740 10 client.0 _ll_register_callbacks cb 0x7f58c402fee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58da17b640 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a21d40 0x55a8c2a26620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58d9179640 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 0x55a8c2a2a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58d997a640 1 -- 172.21.15.31:0/610741361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 msgr2=0x55a8c2a2a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58d997a640 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 0x55a8c2a2a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58d997a640 1 -- 172.21.15.31:0/610741361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a21d40 msgr2=0x55a8c2a26620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58d997a640 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a21d40 0x55a8c2a26620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58d997a640 1 -- 172.21.15.31:0/610741361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c2a26e50 con 0x55a8c2a16b60 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58da17b640 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a21d40 0x55a8c2a26620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58d997a640 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a2a040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f58c4005e30 tx=0x7f58c402fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9faba9c8ffad2b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58b97fa640 1 -- 172.21.15.31:0/610741361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4059060 con 0x55a8c2a16b60 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58b97fa640 1 -- 172.21.15.31:0/610741361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f58c4030640 con 0x55a8c2a16b60 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58ddfb7740 1 -- 172.21.15.31:0/610741361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58cc02b070 con 0x55a8c2a16b60 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58b97fa640 1 -- 172.21.15.31:0/610741361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f58c4060050 con 0x55a8c2a16b60 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58ddfb7740 10 client.15219 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.355+0000 7f58ddfb7740 1 -- 172.21.15.31:0/610741361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a8c2a9d310 con 0x55a8c2a16b60 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.356+0000 7f58b97fa640 1 -- 172.21.15.31:0/610741361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58cc02b070 con 0x55a8c2a16b60 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.356+0000 7f58b97fa640 1 -- 172.21.15.31:0/610741361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f58c40443f0 con 0x55a8c2a16b60 2024-09-16T00:41:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.356+0000 7f58ddfb7740 10 client.15219 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:13.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.356+0000 7f58ddfb7740 10 client.15219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:13.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.356+0000 7f58ddfb7740 20 client.15219 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:13.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.356+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a8c2a9d9d0 0x55a8c2a9fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.356+0000 7f58ddfb7740 4 client.15219 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:41:13.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.356+0000 7f58ddfb7740 1 -- 172.21.15.31:0/610741361 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a8c2983d00 con 0x55a8c2a9d9d0 2024-09-16T00:41:13.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.356+0000 7f58d9179640 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a8c2a9d9d0 0x55a8c2a9fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.357+0000 7f58b97fa640 10 client.15219 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:13.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.357+0000 7f58d9179640 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a8c2a9d9d0 0x55a8c2a9fdf0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0471a80371fda98 server_cookie=473bdf3fba8d5685 in_seq=0 out_seq=0 2024-09-16T00:41:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.357+0000 7f58b97fa640 1 -- 172.21.15.31:0/610741361 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a8c2983d00 con 0x55a8c2a9d9d0 2024-09-16T00:41:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58b97fa640 10 client.15219 handle_command_reply: tid=0 2024-09-16T00:41:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 client.15219 shutdown 2024-09-16T00:41:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 -- 172.21.15.31:0/610741361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 msgr2=0x55a8c2a2a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a2a040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f58c4005e30 tx=0x7f58c402fd50 comp rx=0 tx=0).stop 2024-09-16T00:41:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 -- 172.21.15.31:0/610741361 shutdown_connections 2024-09-16T00:41:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8c2a21d40 0x55a8c2a26620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a8c2a9d9d0 0x55a8c2a9fdf0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8c2a17480 0x55a8c2a2a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 --2- 172.21.15.31:0/610741361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a8c2a16b60 0x55a8c2a2a040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58b97fa640 0 client.15219 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 -- 172.21.15.31:0/610741361 >> 172.21.15.31:0/610741361 conn(0x55a8c2a130f0 msgr2=0x55a8c2a185b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 -- 172.21.15.31:0/610741361 shutdown_connections 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 1 -- 172.21.15.31:0/610741361 wait complete. 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.358+0000 7f58ddfb7740 20 client.15219 trim_cache size 0 max 16384 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:41:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "6145836c-8f4e-4764-acfd-a1f8962b0000", 2024-09-16T00:41:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:41:13.363 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:41:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 -- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58bc0a3240 msgr2=0x7f58bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58bc0a3240 0x7f58bc0a5700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f58cc01d000 tx=0x7f58cc020000 comp rx=0 tx=0).stop 2024-09-16T00:41:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 -- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074f40 msgr2=0x7f58d419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074f40 0x7f58d419ab90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c890 tx=0x7f58c800cd50 comp rx=0 tx=0).stop 2024-09-16T00:41:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 -- 172.21.15.31:0/3191200187 shutdown_connections 2024-09-16T00:41:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58bc0a3240 0x7f58bc0a5700 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58d410e3c0 0x7f58d419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58d4074f40 0x7f58d419ab90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 --2- 172.21.15.31:0/3191200187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58d4074620 0x7f58d419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 -- 172.21.15.31:0/3191200187 >> 172.21.15.31:0/3191200187 conn(0x7f58d40fd8c0 msgr2=0x7f58d40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 -- 172.21.15.31:0/3191200187 shutdown_connections 2024-09-16T00:41:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.359+0000 7f58dbbde640 1 -- 172.21.15.31:0/3191200187 wait complete. 2024-09-16T00:41:13.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.561+0000 7f37317b0640 1 -- 172.21.15.31:0/3179904488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c1036e0 msgr2=0x7f372c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.561+0000 7f37317b0640 1 --2- 172.21.15.31:0/3179904488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c1036e0 0x7f372c107b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3714009930 tx=0x7f371402a420 comp rx=0 tx=0).stop 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.561+0000 7f37317b0640 1 -- 172.21.15.31:0/3179904488 shutdown_connections 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.561+0000 7f37317b0640 1 --2- 172.21.15.31:0/3179904488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f372c1082a0 0x7f372c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.561+0000 7f37317b0640 1 --2- 172.21.15.31:0/3179904488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c1036e0 0x7f372c107b90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.561+0000 7f37317b0640 1 --2- 172.21.15.31:0/3179904488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f372c102d30 0x7f372c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.561+0000 7f37317b0640 1 -- 172.21.15.31:0/3179904488 >> 172.21.15.31:0/3179904488 conn(0x7f372c0fc7d0 msgr2=0x7f372c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.561+0000 7f37317b0640 1 -- 172.21.15.31:0/3179904488 shutdown_connections 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.561+0000 7f37317b0640 1 -- 172.21.15.31:0/3179904488 wait complete. 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f37317b0640 1 Processor -- start 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f37317b0640 1 -- start start 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f37317b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c102d30 0x7f372c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f37317b0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f372c1036e0 0x7f372c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f37317b0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f372c1082a0 0x7f372c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f37317b0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1120a0 con 0x7f372c102d30 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f37317b0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f372c111f20 con 0x7f372c1082a0 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f37317b0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f372c112220 con 0x7f372c1036e0 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f372affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c102d30 0x7f372c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f372affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c102d30 0x7f372c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60420/0 (socket says 172.21.15.31:60420) 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.562+0000 7f372affd640 1 -- 172.21.15.31:0/3995017684 learned_addr learned my addr 172.21.15.31:0/3995017684 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f372b7fe640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f372c1082a0 0x7f372c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f372affd640 1 -- 172.21.15.31:0/3995017684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f372c1036e0 msgr2=0x7f372c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f372a7fc640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f372c1036e0 0x7f372c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f372affd640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f372c1036e0 0x7f372c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f372affd640 1 -- 172.21.15.31:0/3995017684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f372c1082a0 msgr2=0x7f372c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f372affd640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f372c1082a0 0x7f372c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f372affd640 1 -- 172.21.15.31:0/3995017684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c1a6b00 con 0x7f372c102d30 2024-09-16T00:41:13.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f372b7fe640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f372c1082a0 0x7f372c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f372affd640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c102d30 0x7f372c100990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f371c00d950 tx=0x7f371c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f3723fff640 1 -- 172.21.15.31:0/3995017684 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c017070 con 0x7f372c102d30 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f3723fff640 1 -- 172.21.15.31:0/3995017684 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c009c30 con 0x7f372c102d30 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c1a6df0 con 0x7f372c102d30 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f3723fff640 1 -- 172.21.15.31:0/3995017684 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c00f6b0 con 0x7f372c102d30 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.563+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1a72a0 con 0x7f372c102d30 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.564+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372c103110 con 0x7f372c102d30 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.566+0000 7f3723fff640 1 -- 172.21.15.31:0/3995017684 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f371c0109d0 con 0x7f372c102d30 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.566+0000 7f3723fff640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37180a3240 0x7f37180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.566+0000 7f372a7fc640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37180a3240 0x7f37180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.566+0000 7f3723fff640 1 -- 172.21.15.31:0/3995017684 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f371c0cfd10 con 0x7f372c102d30 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.567+0000 7f372a7fc640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37180a3240 0x7f37180a5700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3714009230 tx=0x7f3714000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:13.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.569+0000 7f3723fff640 1 -- 172.21.15.31:0/3995017684 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f371c009de0 con 0x7f372c102d30 2024-09-16T00:41:13.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.743+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f372c10a320 con 0x7f37180a3240 2024-09-16T00:41:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.751+0000 7f3723fff640 1 -- 172.21.15.31:0/3995017684 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340409 (secure 0 0 0) 0x7f3714032db0 con 0x7f37180a3240 2024-09-16T00:41:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:41:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37180a3240 msgr2=0x7f37180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37180a3240 0x7f37180a5700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3714009230 tx=0x7f3714000bc0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c102d30 msgr2=0x7f372c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c102d30 0x7f372c100990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f371c00d950 tx=0x7f371c00de10 comp rx=0 tx=0).stop 2024-09-16T00:41:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 shutdown_connections 2024-09-16T00:41:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f372c1082a0 0x7f372c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f37180a3240 0x7f37180a5700 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f372c1036e0 0x7f372c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 --2- 172.21.15.31:0/3995017684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c102d30 0x7f372c100990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 >> 172.21.15.31:0/3995017684 conn(0x7f372c0fc7d0 msgr2=0x7f372c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 shutdown_connections 2024-09-16T00:41:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:13.756+0000 7f37317b0640 1 -- 172.21.15.31:0/3995017684 wait complete. 2024-09-16T00:41:14.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:41:15.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:14 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:41:15.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:14 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:15.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:14 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:41:15.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:14 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:15.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:14 smithi031 ceph-mon[30956]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:15.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:14 smithi031 ceph-mon[30956]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:15.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:14 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:41:15.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:14 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:15.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:14 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:41:15.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:14 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:14 smithi102 ceph-mon[35364]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:15.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:14 smithi102 ceph-mon[35364]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:15.078 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-16T00:41:13.687747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660841,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":336,"num_read_kb":285,"num_write":277,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":717,"ondisk_log_size":717,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400908,"kb_used_data":14156,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540916,"statfs":{"total":1151940427776,"available":1151529897984,"internally_reserved":0,"allocated":14495744,"data_stored":8185985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":17,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003464"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586150+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:40:59.586150+0000","last_peered":"2024-09-16T00:40:59.586150+0000","last_clean":"2024-09-16T00:40:59.586150+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:40:59.586150+0000","last_undegraded":"2024-09-16T00:40:59.586150+0000","last_fullsized":"2024-09-16T00:40:59.586150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589958+0000","last_change":"2024-09-16T00:40:59.589958+0000","last_active":"2024-09-16T00:40:59.589958+0000","last_peered":"2024-09-16T00:40:59.589958+0000","last_clean":"2024-09-16T00:40:59.589958+0000","last_became_active":"2024-09-16T00:40:59.589549+0000","last_became_peered":"2024-09-16T00:40:59.589549+0000","last_unstale":"2024-09-16T00:40:59.589958+0000","last_undegraded":"2024-09-16T00:40:59.589958+0000","last_fullsized":"2024-09-16T00:40:59.589958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:33.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588133+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:40:59.588133+0000","last_peered":"2024-09-16T00:40:59.588133+0000","last_clean":"2024-09-16T00:40:59.588133+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:40:59.588133+0000","last_undegraded":"2024-09-16T00:40:59.588133+0000","last_fullsized":"2024-09-16T00:40:59.588133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747727+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:40:59.747727+0000","last_peered":"2024-09-16T00:40:59.747727+0000","last_clean":"2024-09-16T00:40:59.747727+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:40:59.747727+0000","last_undegraded":"2024-09-16T00:40:59.747727+0000","last_fullsized":"2024-09-16T00:40:59.747727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162291+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:41:00.162291+0000","last_peered":"2024-09-16T00:41:00.162291+0000","last_clean":"2024-09-16T00:41:00.162291+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:41:00.162291+0000","last_undegraded":"2024-09-16T00:41:00.162291+0000","last_fullsized":"2024-09-16T00:41:00.162291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589935+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:40:59.589935+0000","last_peered":"2024-09-16T00:40:59.589935+0000","last_clean":"2024-09-16T00:40:59.589935+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:40:59.589935+0000","last_undegraded":"2024-09-16T00:40:59.589935+0000","last_fullsized":"2024-09-16T00:40:59.589935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586994+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:40:59.586994+0000","last_peered":"2024-09-16T00:40:59.586994+0000","last_clean":"2024-09-16T00:40:59.586994+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:40:59.586994+0000","last_undegraded":"2024-09-16T00:40:59.586994+0000","last_fullsized":"2024-09-16T00:40:59.586994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587478+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:40:59.587478+0000","last_peered":"2024-09-16T00:40:59.587478+0000","last_clean":"2024-09-16T00:40:59.587478+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:40:59.587478+0000","last_undegraded":"2024-09-16T00:40:59.587478+0000","last_fullsized":"2024-09-16T00:40:59.587478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590172+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:40:59.590172+0000","last_peered":"2024-09-16T00:40:59.590172+0000","last_clean":"2024-09-16T00:40:59.590172+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:40:59.590172+0000","last_undegraded":"2024-09-16T00:40:59.590172+0000","last_fullsized":"2024-09-16T00:40:59.590172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587026+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:40:59.587026+0000","last_peered":"2024-09-16T00:40:59.587026+0000","last_clean":"2024-09-16T00:40:59.587026+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:40:59.587026+0000","last_undegraded":"2024-09-16T00:40:59.587026+0000","last_fullsized":"2024-09-16T00:40:59.587026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.596631+0000","last_change":"2024-09-16T00:40:59.596631+0000","last_active":"2024-09-16T00:40:59.596631+0000","last_peered":"2024-09-16T00:40:59.596631+0000","last_clean":"2024-09-16T00:40:59.596631+0000","last_became_active":"2024-09-16T00:40:59.596165+0000","last_became_peered":"2024-09-16T00:40:59.596165+0000","last_unstale":"2024-09-16T00:40:59.596631+0000","last_undegraded":"2024-09-16T00:40:59.596631+0000","last_fullsized":"2024-09-16T00:40:59.596631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:56.918291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,1],"acting":[9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590072+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:40:59.590072+0000","last_peered":"2024-09-16T00:40:59.590072+0000","last_clean":"2024-09-16T00:40:59.590072+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:40:59.590072+0000","last_undegraded":"2024-09-16T00:40:59.590072+0000","last_fullsized":"2024-09-16T00:40:59.590072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592530+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:40:59.592530+0000","last_peered":"2024-09-16T00:40:59.592530+0000","last_clean":"2024-09-16T00:40:59.592530+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:40:59.592530+0000","last_undegraded":"2024-09-16T00:40:59.592530+0000","last_fullsized":"2024-09-16T00:40:59.592530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586270+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:40:59.586270+0000","last_peered":"2024-09-16T00:40:59.586270+0000","last_clean":"2024-09-16T00:40:59.586270+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:40:59.586270+0000","last_undegraded":"2024-09-16T00:40:59.586270+0000","last_fullsized":"2024-09-16T00:40:59.586270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590233+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:40:59.590233+0000","last_peered":"2024-09-16T00:40:59.590233+0000","last_clean":"2024-09-16T00:40:59.590233+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:40:59.590233+0000","last_undegraded":"2024-09-16T00:40:59.590233+0000","last_fullsized":"2024-09-16T00:40:59.590233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747373+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:40:59.747373+0000","last_peered":"2024-09-16T00:40:59.747373+0000","last_clean":"2024-09-16T00:40:59.747373+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:40:59.747373+0000","last_undegraded":"2024-09-16T00:40:59.747373+0000","last_fullsized":"2024-09-16T00:40:59.747373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747467+0000","last_change":"2024-09-16T00:39:59.591063+0000","last_active":"2024-09-16T00:40:59.747467+0000","last_peered":"2024-09-16T00:40:59.747467+0000","last_clean":"2024-09-16T00:40:59.747467+0000","last_became_active":"2024-09-16T00:39:59.590574+0000","last_became_peered":"2024-09-16T00:39:59.590574+0000","last_unstale":"2024-09-16T00:40:59.747467+0000","last_undegraded":"2024-09-16T00:40:59.747467+0000","last_fullsized":"2024-09-16T00:40:59.747467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:57.020592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590255+0000","last_change":"2024-09-16T00:39:59.595772+0000","last_active":"2024-09-16T00:40:59.590255+0000","last_peered":"2024-09-16T00:40:59.590255+0000","last_clean":"2024-09-16T00:40:59.590255+0000","last_became_active":"2024-09-16T00:39:59.590772+0000","last_became_peered":"2024-09-16T00:39:59.590772+0000","last_unstale":"2024-09-16T00:40:59.590255+0000","last_undegraded":"2024-09-16T00:40:59.590255+0000","last_fullsized":"2024-09-16T00:40:59.590255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:53.236019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747573+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:40:59.747573+0000","last_peered":"2024-09-16T00:40:59.747573+0000","last_clean":"2024-09-16T00:40:59.747573+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:40:59.747573+0000","last_undegraded":"2024-09-16T00:40:59.747573+0000","last_fullsized":"2024-09-16T00:40:59.747573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747545+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:40:59.747545+0000","last_peered":"2024-09-16T00:40:59.747545+0000","last_clean":"2024-09-16T00:40:59.747545+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:40:59.747545+0000","last_undegraded":"2024-09-16T00:40:59.747545+0000","last_fullsized":"2024-09-16T00:40:59.747545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585840+0000","last_change":"2024-09-16T00:39:59.386827+0000","last_active":"2024-09-16T00:40:59.585840+0000","last_peered":"2024-09-16T00:40:59.585840+0000","last_clean":"2024-09-16T00:40:59.585840+0000","last_became_active":"2024-09-16T00:39:59.386389+0000","last_became_peered":"2024-09-16T00:39:59.386389+0000","last_unstale":"2024-09-16T00:40:59.585840+0000","last_undegraded":"2024-09-16T00:40:59.585840+0000","last_fullsized":"2024-09-16T00:40:59.585840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:12:21.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585940+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:40:59.585940+0000","last_peered":"2024-09-16T00:40:59.585940+0000","last_clean":"2024-09-16T00:40:59.585940+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:40:59.585940+0000","last_undegraded":"2024-09-16T00:40:59.585940+0000","last_fullsized":"2024-09-16T00:40:59.585940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587363+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:40:59.587363+0000","last_peered":"2024-09-16T00:40:59.587363+0000","last_clean":"2024-09-16T00:40:59.587363+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:40:59.587363+0000","last_undegraded":"2024-09-16T00:40:59.587363+0000","last_fullsized":"2024-09-16T00:40:59.587363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590175+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:40:59.590175+0000","last_peered":"2024-09-16T00:40:59.590175+0000","last_clean":"2024-09-16T00:40:59.590175+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:40:59.590175+0000","last_undegraded":"2024-09-16T00:40:59.590175+0000","last_fullsized":"2024-09-16T00:40:59.590175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162114+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:41:00.162114+0000","last_peered":"2024-09-16T00:41:00.162114+0000","last_clean":"2024-09-16T00:41:00.162114+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:41:00.162114+0000","last_undegraded":"2024-09-16T00:41:00.162114+0000","last_fullsized":"2024-09-16T00:41:00.162114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590740+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:40:59.590740+0000","last_peered":"2024-09-16T00:40:59.590740+0000","last_clean":"2024-09-16T00:40:59.590740+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:40:59.590740+0000","last_undegraded":"2024-09-16T00:40:59.590740+0000","last_fullsized":"2024-09-16T00:40:59.590740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588415+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:40:59.588415+0000","last_peered":"2024-09-16T00:40:59.588415+0000","last_clean":"2024-09-16T00:40:59.588415+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:40:59.588415+0000","last_undegraded":"2024-09-16T00:40:59.588415+0000","last_fullsized":"2024-09-16T00:40:59.588415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747653+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:40:59.747653+0000","last_peered":"2024-09-16T00:40:59.747653+0000","last_clean":"2024-09-16T00:40:59.747653+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:40:59.747653+0000","last_undegraded":"2024-09-16T00:40:59.747653+0000","last_fullsized":"2024-09-16T00:40:59.747653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:04.350207+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:41:04.350207+0000","last_peered":"2024-09-16T00:41:04.350207+0000","last_clean":"2024-09-16T00:41:04.350207+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:41:04.350207+0000","last_undegraded":"2024-09-16T00:41:04.350207+0000","last_fullsized":"2024-09-16T00:41:04.350207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747093+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:40:59.747093+0000","last_peered":"2024-09-16T00:40:59.747093+0000","last_clean":"2024-09-16T00:40:59.747093+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:40:59.747093+0000","last_undegraded":"2024-09-16T00:40:59.747093+0000","last_fullsized":"2024-09-16T00:40:59.747093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595527+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:40:59.595527+0000","last_peered":"2024-09-16T00:40:59.595527+0000","last_clean":"2024-09-16T00:40:59.595527+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:40:59.595527+0000","last_undegraded":"2024-09-16T00:40:59.595527+0000","last_fullsized":"2024-09-16T00:40:59.595527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587025+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:40:59.587025+0000","last_peered":"2024-09-16T00:40:59.587025+0000","last_clean":"2024-09-16T00:40:59.587025+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:40:59.587025+0000","last_undegraded":"2024-09-16T00:40:59.587025+0000","last_fullsized":"2024-09-16T00:40:59.587025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747410+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:40:59.747410+0000","last_peered":"2024-09-16T00:40:59.747410+0000","last_clean":"2024-09-16T00:40:59.747410+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:40:59.747410+0000","last_undegraded":"2024-09-16T00:40:59.747410+0000","last_fullsized":"2024-09-16T00:40:59.747410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747586+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:40:59.747586+0000","last_peered":"2024-09-16T00:40:59.747586+0000","last_clean":"2024-09-16T00:40:59.747586+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:40:59.747586+0000","last_undegraded":"2024-09-16T00:40:59.747586+0000","last_fullsized":"2024-09-16T00:40:59.747586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585296+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:40:59.585296+0000","last_peered":"2024-09-16T00:40:59.585296+0000","last_clean":"2024-09-16T00:40:59.585296+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:40:59.585296+0000","last_undegraded":"2024-09-16T00:40:59.585296+0000","last_fullsized":"2024-09-16T00:40:59.585296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586050+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:40:59.586050+0000","last_peered":"2024-09-16T00:40:59.586050+0000","last_clean":"2024-09-16T00:40:59.586050+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:40:59.586050+0000","last_undegraded":"2024-09-16T00:40:59.586050+0000","last_fullsized":"2024-09-16T00:40:59.586050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747487+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:40:59.747487+0000","last_peered":"2024-09-16T00:40:59.747487+0000","last_clean":"2024-09-16T00:40:59.747487+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:40:59.747487+0000","last_undegraded":"2024-09-16T00:40:59.747487+0000","last_fullsized":"2024-09-16T00:40:59.747487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592602+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:40:59.592602+0000","last_peered":"2024-09-16T00:40:59.592602+0000","last_clean":"2024-09-16T00:40:59.592602+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:40:59.592602+0000","last_undegraded":"2024-09-16T00:40:59.592602+0000","last_fullsized":"2024-09-16T00:40:59.592602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595268+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:40:59.595268+0000","last_peered":"2024-09-16T00:40:59.595268+0000","last_clean":"2024-09-16T00:40:59.595268+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:40:59.595268+0000","last_undegraded":"2024-09-16T00:40:59.595268+0000","last_fullsized":"2024-09-16T00:40:59.595268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.591482+0000","last_change":"2024-09-16T00:40:59.591482+0000","last_active":"2024-09-16T00:40:59.591482+0000","last_peered":"2024-09-16T00:40:59.591482+0000","last_clean":"2024-09-16T00:40:59.591482+0000","last_became_active":"2024-09-16T00:40:59.590978+0000","last_became_peered":"2024-09-16T00:40:59.590978+0000","last_unstale":"2024-09-16T00:40:59.591482+0000","last_undegraded":"2024-09-16T00:40:59.591482+0000","last_fullsized":"2024-09-16T00:40:59.591482+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:56.975117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594993+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:40:59.594993+0000","last_peered":"2024-09-16T00:40:59.594993+0000","last_clean":"2024-09-16T00:40:59.594993+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:40:59.594993+0000","last_undegraded":"2024-09-16T00:40:59.594993+0000","last_fullsized":"2024-09-16T00:40:59.594993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595391+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:40:59.595391+0000","last_peered":"2024-09-16T00:40:59.595391+0000","last_clean":"2024-09-16T00:40:59.595391+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:40:59.595391+0000","last_undegraded":"2024-09-16T00:40:59.595391+0000","last_fullsized":"2024-09-16T00:40:59.595391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594855+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:40:59.594855+0000","last_peered":"2024-09-16T00:40:59.594855+0000","last_clean":"2024-09-16T00:40:59.594855+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:40:59.594855+0000","last_undegraded":"2024-09-16T00:40:59.594855+0000","last_fullsized":"2024-09-16T00:40:59.594855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747209+0000","last_change":"2024-09-16T00:39:59.376963+0000","last_active":"2024-09-16T00:40:59.747209+0000","last_peered":"2024-09-16T00:40:59.747209+0000","last_clean":"2024-09-16T00:40:59.747209+0000","last_became_active":"2024-09-16T00:39:59.376538+0000","last_became_peered":"2024-09-16T00:39:59.376538+0000","last_unstale":"2024-09-16T00:40:59.747209+0000","last_undegraded":"2024-09-16T00:40:59.747209+0000","last_fullsized":"2024-09-16T00:40:59.747209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:02.077749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587907+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:40:59.587907+0000","last_peered":"2024-09-16T00:40:59.587907+0000","last_clean":"2024-09-16T00:40:59.587907+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:40:59.587907+0000","last_undegraded":"2024-09-16T00:40:59.587907+0000","last_fullsized":"2024-09-16T00:40:59.587907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747729+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:40:59.747729+0000","last_peered":"2024-09-16T00:40:59.747729+0000","last_clean":"2024-09-16T00:40:59.747729+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:40:59.747729+0000","last_undegraded":"2024-09-16T00:40:59.747729+0000","last_fullsized":"2024-09-16T00:40:59.747729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590591+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:40:59.590591+0000","last_peered":"2024-09-16T00:40:59.590591+0000","last_clean":"2024-09-16T00:40:59.590591+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:40:59.590591+0000","last_undegraded":"2024-09-16T00:40:59.590591+0000","last_fullsized":"2024-09-16T00:40:59.590591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592357+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:40:59.592357+0000","last_peered":"2024-09-16T00:40:59.592357+0000","last_clean":"2024-09-16T00:40:59.592357+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:40:59.592357+0000","last_undegraded":"2024-09-16T00:40:59.592357+0000","last_fullsized":"2024-09-16T00:40:59.592357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592743+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:40:59.592743+0000","last_peered":"2024-09-16T00:40:59.592743+0000","last_clean":"2024-09-16T00:40:59.592743+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:40:59.592743+0000","last_undegraded":"2024-09-16T00:40:59.592743+0000","last_fullsized":"2024-09-16T00:40:59.592743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592923+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:40:59.592923+0000","last_peered":"2024-09-16T00:40:59.592923+0000","last_clean":"2024-09-16T00:40:59.592923+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:40:59.592923+0000","last_undegraded":"2024-09-16T00:40:59.592923+0000","last_fullsized":"2024-09-16T00:40:59.592923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589988+0000","last_change":"2024-09-16T00:39:59.593442+0000","last_active":"2024-09-16T00:40:59.589988+0000","last_peered":"2024-09-16T00:40:59.589988+0000","last_clean":"2024-09-16T00:40:59.589988+0000","last_became_active":"2024-09-16T00:39:59.593039+0000","last_became_peered":"2024-09-16T00:39:59.593039+0000","last_unstale":"2024-09-16T00:40:59.589988+0000","last_undegraded":"2024-09-16T00:40:59.589988+0000","last_fullsized":"2024-09-16T00:40:59.589988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:36.644148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587371+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:40:59.587371+0000","last_peered":"2024-09-16T00:40:59.587371+0000","last_clean":"2024-09-16T00:40:59.587371+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:40:59.587371+0000","last_undegraded":"2024-09-16T00:40:59.587371+0000","last_fullsized":"2024-09-16T00:40:59.587371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592697+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:40:59.592697+0000","last_peered":"2024-09-16T00:40:59.592697+0000","last_clean":"2024-09-16T00:40:59.592697+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:40:59.592697+0000","last_undegraded":"2024-09-16T00:40:59.592697+0000","last_fullsized":"2024-09-16T00:40:59.592697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589435+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:40:59.589435+0000","last_peered":"2024-09-16T00:40:59.589435+0000","last_clean":"2024-09-16T00:40:59.589435+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:40:59.589435+0000","last_undegraded":"2024-09-16T00:40:59.589435+0000","last_fullsized":"2024-09-16T00:40:59.589435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161277+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:41:00.161277+0000","last_peered":"2024-09-16T00:41:00.161277+0000","last_clean":"2024-09-16T00:41:00.161277+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:41:00.161277+0000","last_undegraded":"2024-09-16T00:41:00.161277+0000","last_fullsized":"2024-09-16T00:41:00.161277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592993+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:40:59.592993+0000","last_peered":"2024-09-16T00:40:59.592993+0000","last_clean":"2024-09-16T00:40:59.592993+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:40:59.592993+0000","last_undegraded":"2024-09-16T00:40:59.592993+0000","last_fullsized":"2024-09-16T00:40:59.592993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595517+0000","last_change":"2024-09-16T00:39:59.594994+0000","last_active":"2024-09-16T00:40:59.595517+0000","last_peered":"2024-09-16T00:40:59.595517+0000","last_clean":"2024-09-16T00:40:59.595517+0000","last_became_active":"2024-09-16T00:39:59.590640+0000","last_became_peered":"2024-09-16T00:39:59.590640+0000","last_unstale":"2024-09-16T00:40:59.595517+0000","last_undegraded":"2024-09-16T00:40:59.595517+0000","last_fullsized":"2024-09-16T00:40:59.595517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:21.391494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587245+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:40:59.587245+0000","last_peered":"2024-09-16T00:40:59.587245+0000","last_clean":"2024-09-16T00:40:59.587245+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:40:59.587245+0000","last_undegraded":"2024-09-16T00:40:59.587245+0000","last_fullsized":"2024-09-16T00:40:59.587245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747566+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:40:59.747566+0000","last_peered":"2024-09-16T00:40:59.747566+0000","last_clean":"2024-09-16T00:40:59.747566+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:40:59.747566+0000","last_undegraded":"2024-09-16T00:40:59.747566+0000","last_fullsized":"2024-09-16T00:40:59.747566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586785+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:40:59.586785+0000","last_peered":"2024-09-16T00:40:59.586785+0000","last_clean":"2024-09-16T00:40:59.586785+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:40:59.586785+0000","last_undegraded":"2024-09-16T00:40:59.586785+0000","last_fullsized":"2024-09-16T00:40:59.586785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588117+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:40:59.588117+0000","last_peered":"2024-09-16T00:40:59.588117+0000","last_clean":"2024-09-16T00:40:59.588117+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:40:59.588117+0000","last_undegraded":"2024-09-16T00:40:59.588117+0000","last_fullsized":"2024-09-16T00:40:59.588117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588295+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:40:59.588295+0000","last_peered":"2024-09-16T00:40:59.588295+0000","last_clean":"2024-09-16T00:40:59.588295+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:40:59.588295+0000","last_undegraded":"2024-09-16T00:40:59.588295+0000","last_fullsized":"2024-09-16T00:40:59.588295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585921+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:40:59.585921+0000","last_peered":"2024-09-16T00:40:59.585921+0000","last_clean":"2024-09-16T00:40:59.585921+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:40:59.585921+0000","last_undegraded":"2024-09-16T00:40:59.585921+0000","last_fullsized":"2024-09-16T00:40:59.585921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595364+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:40:59.595364+0000","last_peered":"2024-09-16T00:40:59.595364+0000","last_clean":"2024-09-16T00:40:59.595364+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:40:59.595364+0000","last_undegraded":"2024-09-16T00:40:59.595364+0000","last_fullsized":"2024-09-16T00:40:59.595364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592245+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:40:59.592245+0000","last_peered":"2024-09-16T00:40:59.592245+0000","last_clean":"2024-09-16T00:40:59.592245+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:40:59.592245+0000","last_undegraded":"2024-09-16T00:40:59.592245+0000","last_fullsized":"2024-09-16T00:40:59.592245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590466+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:40:59.590466+0000","last_peered":"2024-09-16T00:40:59.590466+0000","last_clean":"2024-09-16T00:40:59.590466+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:40:59.590466+0000","last_undegraded":"2024-09-16T00:40:59.590466+0000","last_fullsized":"2024-09-16T00:40:59.590466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586021+0000","last_change":"2024-09-16T00:39:59.593568+0000","last_active":"2024-09-16T00:40:59.586021+0000","last_peered":"2024-09-16T00:40:59.586021+0000","last_clean":"2024-09-16T00:40:59.586021+0000","last_became_active":"2024-09-16T00:39:59.593209+0000","last_became_peered":"2024-09-16T00:39:59.593209+0000","last_unstale":"2024-09-16T00:40:59.586021+0000","last_undegraded":"2024-09-16T00:40:59.586021+0000","last_fullsized":"2024-09-16T00:40:59.586021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:56.516773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.597069+0000","last_change":"2024-09-16T00:40:59.596544+0000","last_active":"2024-09-16T00:40:59.597069+0000","last_peered":"2024-09-16T00:40:59.597069+0000","last_clean":"2024-09-16T00:40:59.597069+0000","last_became_active":"2024-09-16T00:40:59.596128+0000","last_became_peered":"2024-09-16T00:40:59.596128+0000","last_unstale":"2024-09-16T00:40:59.597069+0000","last_undegraded":"2024-09-16T00:40:59.597069+0000","last_fullsized":"2024-09-16T00:40:59.597069+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:22.691164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587694+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:40:59.587694+0000","last_peered":"2024-09-16T00:40:59.587694+0000","last_clean":"2024-09-16T00:40:59.587694+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:40:59.587694+0000","last_undegraded":"2024-09-16T00:40:59.587694+0000","last_fullsized":"2024-09-16T00:40:59.587694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747362+0000","last_change":"2024-09-16T00:39:59.380258+0000","last_active":"2024-09-16T00:40:59.747362+0000","last_peered":"2024-09-16T00:40:59.747362+0000","last_clean":"2024-09-16T00:40:59.747362+0000","last_became_active":"2024-09-16T00:39:59.379813+0000","last_became_peered":"2024-09-16T00:39:59.379813+0000","last_unstale":"2024-09-16T00:40:59.747362+0000","last_undegraded":"2024-09-16T00:40:59.747362+0000","last_fullsized":"2024-09-16T00:40:59.747362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:03.110305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595116+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:40:59.595116+0000","last_peered":"2024-09-16T00:40:59.595116+0000","last_clean":"2024-09-16T00:40:59.595116+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:40:59.595116+0000","last_undegraded":"2024-09-16T00:40:59.595116+0000","last_fullsized":"2024-09-16T00:40:59.595116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588686+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:40:59.588686+0000","last_peered":"2024-09-16T00:40:59.588686+0000","last_clean":"2024-09-16T00:40:59.588686+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:40:59.588686+0000","last_undegraded":"2024-09-16T00:40:59.588686+0000","last_fullsized":"2024-09-16T00:40:59.588686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588982+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:40:59.588982+0000","last_peered":"2024-09-16T00:40:59.588982+0000","last_clean":"2024-09-16T00:40:59.588982+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:40:59.588982+0000","last_undegraded":"2024-09-16T00:40:59.588982+0000","last_fullsized":"2024-09-16T00:40:59.588982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586881+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:40:59.586881+0000","last_peered":"2024-09-16T00:40:59.586881+0000","last_clean":"2024-09-16T00:40:59.586881+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:40:59.586881+0000","last_undegraded":"2024-09-16T00:40:59.586881+0000","last_fullsized":"2024-09-16T00:40:59.586881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":278,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:10.298879+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:41:10.298879+0000","last_peered":"2024-09-16T00:41:10.298879+0000","last_clean":"2024-09-16T00:41:10.298879+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:41:10.298879+0000","last_undegraded":"2024-09-16T00:41:10.298879+0000","last_fullsized":"2024-09-16T00:41:10.298879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69441,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":204,"num_read_kb":170,"num_write":35,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585067+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:40:59.585067+0000","last_peered":"2024-09-16T00:40:59.585067+0000","last_clean":"2024-09-16T00:40:59.585067+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:40:59.585067+0000","last_undegraded":"2024-09-16T00:40:59.585067+0000","last_fullsized":"2024-09-16T00:40:59.585067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587571+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:40:59.587571+0000","last_peered":"2024-09-16T00:40:59.587571+0000","last_clean":"2024-09-16T00:40:59.587571+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:40:59.587571+0000","last_undegraded":"2024-09-16T00:40:59.587571+0000","last_fullsized":"2024-09-16T00:40:59.587571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589244+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:40:59.589244+0000","last_peered":"2024-09-16T00:40:59.589244+0000","last_clean":"2024-09-16T00:40:59.589244+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:40:59.589244+0000","last_undegraded":"2024-09-16T00:40:59.589244+0000","last_fullsized":"2024-09-16T00:40:59.589244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590297+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:40:59.590297+0000","last_peered":"2024-09-16T00:40:59.590297+0000","last_clean":"2024-09-16T00:40:59.590297+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:40:59.590297+0000","last_undegraded":"2024-09-16T00:40:59.590297+0000","last_fullsized":"2024-09-16T00:40:59.590297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162296+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:41:00.162296+0000","last_peered":"2024-09-16T00:41:00.162296+0000","last_clean":"2024-09-16T00:41:00.162296+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:41:00.162296+0000","last_undegraded":"2024-09-16T00:41:00.162296+0000","last_fullsized":"2024-09-16T00:41:00.162296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592831+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:40:59.592831+0000","last_peered":"2024-09-16T00:40:59.592831+0000","last_clean":"2024-09-16T00:40:59.592831+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:40:59.592831+0000","last_undegraded":"2024-09-16T00:40:59.592831+0000","last_fullsized":"2024-09-16T00:40:59.592831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584869+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:40:59.584869+0000","last_peered":"2024-09-16T00:40:59.584869+0000","last_clean":"2024-09-16T00:40:59.584869+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:40:59.584869+0000","last_undegraded":"2024-09-16T00:40:59.584869+0000","last_fullsized":"2024-09-16T00:40:59.584869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587926+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:40:59.587926+0000","last_peered":"2024-09-16T00:40:59.587926+0000","last_clean":"2024-09-16T00:40:59.587926+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:40:59.587926+0000","last_undegraded":"2024-09-16T00:40:59.587926+0000","last_fullsized":"2024-09-16T00:40:59.587926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747739+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:40:59.747739+0000","last_peered":"2024-09-16T00:40:59.747739+0000","last_clean":"2024-09-16T00:40:59.747739+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:40:59.747739+0000","last_undegraded":"2024-09-16T00:40:59.747739+0000","last_fullsized":"2024-09-16T00:40:59.747739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747139+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:40:59.747139+0000","last_peered":"2024-09-16T00:40:59.747139+0000","last_clean":"2024-09-16T00:40:59.747139+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:40:59.747139+0000","last_undegraded":"2024-09-16T00:40:59.747139+0000","last_fullsized":"2024-09-16T00:40:59.747139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161443+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:41:00.161443+0000","last_peered":"2024-09-16T00:41:00.161443+0000","last_clean":"2024-09-16T00:41:00.161443+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:41:00.161443+0000","last_undegraded":"2024-09-16T00:41:00.161443+0000","last_fullsized":"2024-09-16T00:41:00.161443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584969+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:40:59.584969+0000","last_peered":"2024-09-16T00:40:59.584969+0000","last_clean":"2024-09-16T00:40:59.584969+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:40:59.584969+0000","last_undegraded":"2024-09-16T00:40:59.584969+0000","last_fullsized":"2024-09-16T00:40:59.584969+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588578+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:40:59.588578+0000","last_peered":"2024-09-16T00:40:59.588578+0000","last_clean":"2024-09-16T00:40:59.588578+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:40:59.588578+0000","last_undegraded":"2024-09-16T00:40:59.588578+0000","last_fullsized":"2024-09-16T00:40:59.588578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747585+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:40:59.747585+0000","last_peered":"2024-09-16T00:40:59.747585+0000","last_clean":"2024-09-16T00:40:59.747585+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:40:59.747585+0000","last_undegraded":"2024-09-16T00:40:59.747585+0000","last_fullsized":"2024-09-16T00:40:59.747585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747194+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:40:59.747194+0000","last_peered":"2024-09-16T00:40:59.747194+0000","last_clean":"2024-09-16T00:40:59.747194+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:40:59.747194+0000","last_undegraded":"2024-09-16T00:40:59.747194+0000","last_fullsized":"2024-09-16T00:40:59.747194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594858+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:40:59.594858+0000","last_peered":"2024-09-16T00:40:59.594858+0000","last_clean":"2024-09-16T00:40:59.594858+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:40:59.594858+0000","last_undegraded":"2024-09-16T00:40:59.594858+0000","last_fullsized":"2024-09-16T00:40:59.594858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747376+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:40:59.747376+0000","last_peered":"2024-09-16T00:40:59.747376+0000","last_clean":"2024-09-16T00:40:59.747376+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:40:59.747376+0000","last_undegraded":"2024-09-16T00:40:59.747376+0000","last_fullsized":"2024-09-16T00:40:59.747376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588250+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:40:59.588250+0000","last_peered":"2024-09-16T00:40:59.588250+0000","last_clean":"2024-09-16T00:40:59.588250+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:40:59.588250+0000","last_undegraded":"2024-09-16T00:40:59.588250+0000","last_fullsized":"2024-09-16T00:40:59.588250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587684+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:40:59.587684+0000","last_peered":"2024-09-16T00:40:59.587684+0000","last_clean":"2024-09-16T00:40:59.587684+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:40:59.587684+0000","last_undegraded":"2024-09-16T00:40:59.587684+0000","last_fullsized":"2024-09-16T00:40:59.587684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588368+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:40:59.588368+0000","last_peered":"2024-09-16T00:40:59.588368+0000","last_clean":"2024-09-16T00:40:59.588368+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:40:59.588368+0000","last_undegraded":"2024-09-16T00:40:59.588368+0000","last_fullsized":"2024-09-16T00:40:59.588368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586131+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:40:59.586131+0000","last_peered":"2024-09-16T00:40:59.586131+0000","last_clean":"2024-09-16T00:40:59.586131+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:40:59.586131+0000","last_undegraded":"2024-09-16T00:40:59.586131+0000","last_fullsized":"2024-09-16T00:40:59.586131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588144+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:40:59.588144+0000","last_peered":"2024-09-16T00:40:59.588144+0000","last_clean":"2024-09-16T00:40:59.588144+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:40:59.588144+0000","last_undegraded":"2024-09-16T00:40:59.588144+0000","last_fullsized":"2024-09-16T00:40:59.588144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747412+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:40:59.747412+0000","last_peered":"2024-09-16T00:40:59.747412+0000","last_clean":"2024-09-16T00:40:59.747412+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:40:59.747412+0000","last_undegraded":"2024-09-16T00:40:59.747412+0000","last_fullsized":"2024-09-16T00:40:59.747412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588676+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:40:59.588676+0000","last_peered":"2024-09-16T00:40:59.588676+0000","last_clean":"2024-09-16T00:40:59.588676+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:40:59.588676+0000","last_undegraded":"2024-09-16T00:40:59.588676+0000","last_fullsized":"2024-09-16T00:40:59.588676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587600+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:40:59.587600+0000","last_peered":"2024-09-16T00:40:59.587600+0000","last_clean":"2024-09-16T00:40:59.587600+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:40:59.587600+0000","last_undegraded":"2024-09-16T00:40:59.587600+0000","last_fullsized":"2024-09-16T00:40:59.587600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162004+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:41:00.162004+0000","last_peered":"2024-09-16T00:41:00.162004+0000","last_clean":"2024-09-16T00:41:00.162004+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:41:00.162004+0000","last_undegraded":"2024-09-16T00:41:00.162004+0000","last_fullsized":"2024-09-16T00:41:00.162004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595476+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:40:59.595476+0000","last_peered":"2024-09-16T00:40:59.595476+0000","last_clean":"2024-09-16T00:40:59.595476+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:40:59.595476+0000","last_undegraded":"2024-09-16T00:40:59.595476+0000","last_fullsized":"2024-09-16T00:40:59.595476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747786+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:40:59.747786+0000","last_peered":"2024-09-16T00:40:59.747786+0000","last_clean":"2024-09-16T00:40:59.747786+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:40:59.747786+0000","last_undegraded":"2024-09-16T00:40:59.747786+0000","last_fullsized":"2024-09-16T00:40:59.747786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588960+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:40:59.588960+0000","last_peered":"2024-09-16T00:40:59.588960+0000","last_clean":"2024-09-16T00:40:59.588960+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:40:59.588960+0000","last_undegraded":"2024-09-16T00:40:59.588960+0000","last_fullsized":"2024-09-16T00:40:59.588960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.748038+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:40:59.748038+0000","last_peered":"2024-09-16T00:40:59.748038+0000","last_clean":"2024-09-16T00:40:59.748038+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:40:59.748038+0000","last_undegraded":"2024-09-16T00:40:59.748038+0000","last_fullsized":"2024-09-16T00:40:59.748038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587358+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:40:59.587358+0000","last_peered":"2024-09-16T00:40:59.587358+0000","last_clean":"2024-09-16T00:40:59.587358+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:40:59.587358+0000","last_undegraded":"2024-09-16T00:40:59.587358+0000","last_fullsized":"2024-09-16T00:40:59.587358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747301+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:40:59.747301+0000","last_peered":"2024-09-16T00:40:59.747301+0000","last_clean":"2024-09-16T00:40:59.747301+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:40:59.747301+0000","last_undegraded":"2024-09-16T00:40:59.747301+0000","last_fullsized":"2024-09-16T00:40:59.747301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587542+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:40:59.587542+0000","last_peered":"2024-09-16T00:40:59.587542+0000","last_clean":"2024-09-16T00:40:59.587542+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:40:59.587542+0000","last_undegraded":"2024-09-16T00:40:59.587542+0000","last_fullsized":"2024-09-16T00:40:59.587542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595665+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:40:59.595665+0000","last_peered":"2024-09-16T00:40:59.595665+0000","last_clean":"2024-09-16T00:40:59.595665+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:40:59.595665+0000","last_undegraded":"2024-09-16T00:40:59.595665+0000","last_fullsized":"2024-09-16T00:40:59.595665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747529+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:40:59.747529+0000","last_peered":"2024-09-16T00:40:59.747529+0000","last_clean":"2024-09-16T00:40:59.747529+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:40:59.747529+0000","last_undegraded":"2024-09-16T00:40:59.747529+0000","last_fullsized":"2024-09-16T00:40:59.747529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588319+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:40:59.588319+0000","last_peered":"2024-09-16T00:40:59.588319+0000","last_clean":"2024-09-16T00:40:59.588319+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:40:59.588319+0000","last_undegraded":"2024-09-16T00:40:59.588319+0000","last_fullsized":"2024-09-16T00:40:59.588319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586045+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:40:59.586045+0000","last_peered":"2024-09-16T00:40:59.586045+0000","last_clean":"2024-09-16T00:40:59.586045+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:40:59.586045+0000","last_undegraded":"2024-09-16T00:40:59.586045+0000","last_fullsized":"2024-09-16T00:40:59.586045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747774+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:40:59.747774+0000","last_peered":"2024-09-16T00:40:59.747774+0000","last_clean":"2024-09-16T00:40:59.747774+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:40:59.747774+0000","last_undegraded":"2024-09-16T00:40:59.747774+0000","last_fullsized":"2024-09-16T00:40:59.747774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587142+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:40:59.587142+0000","last_peered":"2024-09-16T00:40:59.587142+0000","last_clean":"2024-09-16T00:40:59.587142+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:40:59.587142+0000","last_undegraded":"2024-09-16T00:40:59.587142+0000","last_fullsized":"2024-09-16T00:40:59.587142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161639+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:41:00.161639+0000","last_peered":"2024-09-16T00:41:00.161639+0000","last_clean":"2024-09-16T00:41:00.161639+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:41:00.161639+0000","last_undegraded":"2024-09-16T00:41:00.161639+0000","last_fullsized":"2024-09-16T00:41:00.161639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161754+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:41:00.161754+0000","last_peered":"2024-09-16T00:41:00.161754+0000","last_clean":"2024-09-16T00:41:00.161754+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:41:00.161754+0000","last_undegraded":"2024-09-16T00:41:00.161754+0000","last_fullsized":"2024-09-16T00:41:00.161754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595267+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:40:59.595267+0000","last_peered":"2024-09-16T00:40:59.595267+0000","last_clean":"2024-09-16T00:40:59.595267+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:40:59.595267+0000","last_undegraded":"2024-09-16T00:40:59.595267+0000","last_fullsized":"2024-09-16T00:40:59.595267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590469+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:40:59.590469+0000","last_peered":"2024-09-16T00:40:59.590469+0000","last_clean":"2024-09-16T00:40:59.590469+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:40:59.590469+0000","last_undegraded":"2024-09-16T00:40:59.590469+0000","last_fullsized":"2024-09-16T00:40:59.590469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592840+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:40:59.592840+0000","last_peered":"2024-09-16T00:40:59.592840+0000","last_clean":"2024-09-16T00:40:59.592840+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:40:59.592840+0000","last_undegraded":"2024-09-16T00:40:59.592840+0000","last_fullsized":"2024-09-16T00:40:59.592840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747793+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:40:59.747793+0000","last_peered":"2024-09-16T00:40:59.747793+0000","last_clean":"2024-09-16T00:40:59.747793+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:40:59.747793+0000","last_undegraded":"2024-09-16T00:40:59.747793+0000","last_fullsized":"2024-09-16T00:40:59.747793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587933+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:40:59.587933+0000","last_peered":"2024-09-16T00:40:59.587933+0000","last_clean":"2024-09-16T00:40:59.587933+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:40:59.587933+0000","last_undegraded":"2024-09-16T00:40:59.587933+0000","last_fullsized":"2024-09-16T00:40:59.587933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747138+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:40:59.747138+0000","last_peered":"2024-09-16T00:40:59.747138+0000","last_clean":"2024-09-16T00:40:59.747138+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:40:59.747138+0000","last_undegraded":"2024-09-16T00:40:59.747138+0000","last_fullsized":"2024-09-16T00:40:59.747138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592328+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:40:59.592328+0000","last_peered":"2024-09-16T00:40:59.592328+0000","last_clean":"2024-09-16T00:40:59.592328+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:40:59.592328+0000","last_undegraded":"2024-09-16T00:40:59.592328+0000","last_fullsized":"2024-09-16T00:40:59.592328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747364+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:40:59.747364+0000","last_peered":"2024-09-16T00:40:59.747364+0000","last_clean":"2024-09-16T00:40:59.747364+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:40:59.747364+0000","last_undegraded":"2024-09-16T00:40:59.747364+0000","last_fullsized":"2024-09-16T00:40:59.747364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586035+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:40:59.586035+0000","last_peered":"2024-09-16T00:40:59.586035+0000","last_clean":"2024-09-16T00:40:59.586035+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:40:59.586035+0000","last_undegraded":"2024-09-16T00:40:59.586035+0000","last_fullsized":"2024-09-16T00:40:59.586035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590395+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:40:59.590395+0000","last_peered":"2024-09-16T00:40:59.590395+0000","last_clean":"2024-09-16T00:40:59.590395+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:40:59.590395+0000","last_undegraded":"2024-09-16T00:40:59.590395+0000","last_fullsized":"2024-09-16T00:40:59.590395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162106+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:41:00.162106+0000","last_peered":"2024-09-16T00:41:00.162106+0000","last_clean":"2024-09-16T00:41:00.162106+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:41:00.162106+0000","last_undegraded":"2024-09-16T00:41:00.162106+0000","last_fullsized":"2024-09-16T00:41:00.162106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162139+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:41:00.162139+0000","last_peered":"2024-09-16T00:41:00.162139+0000","last_clean":"2024-09-16T00:41:00.162139+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:41:00.162139+0000","last_undegraded":"2024-09-16T00:41:00.162139+0000","last_fullsized":"2024-09-16T00:41:00.162139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747779+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:40:59.747779+0000","last_peered":"2024-09-16T00:40:59.747779+0000","last_clean":"2024-09-16T00:40:59.747779+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:40:59.747779+0000","last_undegraded":"2024-09-16T00:40:59.747779+0000","last_fullsized":"2024-09-16T00:40:59.747779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70239,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":210,"num_read_kb":176,"num_write":94,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":210717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547242,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":518379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776175,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1720320,"data_stored":1178382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939699,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":518961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168634,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":518983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1638400,"data_stored":1113425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":514283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1642496,"data_stored":1108929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":518401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":514387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444120,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":583758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":583724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":514373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:41:15.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:41:15.108 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:15.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:14 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:41:15.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:14 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:15.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:14 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:41:15.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:14 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:15.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:14 smithi081 ceph-mon[35539]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:15.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:14 smithi081 ceph-mon[35539]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.568+0000 7f79f84f9640 1 -- 172.21.15.31:0/3491742985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0102790 msgr2=0x7f79f01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.568+0000 7f79f84f9640 1 --2- 172.21.15.31:0/3491742985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0102790 0x7f79f01067e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e002a410 comp rx=0 tx=0).stop 2024-09-16T00:41:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.568+0000 7f79f84f9640 1 -- 172.21.15.31:0/3491742985 shutdown_connections 2024-09-16T00:41:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.568+0000 7f79f84f9640 1 --2- 172.21.15.31:0/3491742985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79f0106f10 0x7f79f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.568+0000 7f79f84f9640 1 --2- 172.21.15.31:0/3491742985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0102790 0x7f79f01067e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.568+0000 7f79f84f9640 1 --2- 172.21.15.31:0/3491742985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f0101de0 0x7f79f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.568+0000 7f79f84f9640 1 -- 172.21.15.31:0/3491742985 >> 172.21.15.31:0/3491742985 conn(0x7f79f00fd8c0 msgr2=0x7f79f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.568+0000 7f79f84f9640 1 -- 172.21.15.31:0/3491742985 shutdown_connections 2024-09-16T00:41:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.568+0000 7f79f84f9640 1 -- 172.21.15.31:0/3491742985 wait complete. 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f84f9640 1 Processor -- start 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f84f9640 1 -- start start 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f84f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0101de0 0x7f79f01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f84f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f0102790 0x7f79f01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f84f9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79f0106f10 0x7f79f019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f84f9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111f70 con 0x7f79f0101de0 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f84f9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111df0 con 0x7f79f0106f10 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f84f9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01120f0 con 0x7f79f0102790 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f6295640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0101de0 0x7f79f01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f6295640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0101de0 0x7f79f01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60442/0 (socket says 172.21.15.31:60442) 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f6295640 1 -- 172.21.15.31:0/737538572 learned_addr learned my addr 172.21.15.31:0/737538572 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f5a94640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f0102790 0x7f79f01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.569+0000 7f79f6a96640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79f0106f10 0x7f79f019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79f6295640 1 -- 172.21.15.31:0/737538572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f0102790 msgr2=0x7f79f01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79f6295640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f0102790 0x7f79f01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79f6295640 1 -- 172.21.15.31:0/737538572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79f0106f10 msgr2=0x7f79f019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79f6295640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79f0106f10 0x7f79f019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79f6295640 1 -- 172.21.15.31:0/737538572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f019d460 con 0x7f79f0101de0 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79f6295640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0101de0 0x7f79f01984b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c970 tx=0x7f79ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79e77fe640 1 -- 172.21.15.31:0/737538572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec007bb0 con 0x7f79f0101de0 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79f84f9640 1 -- 172.21.15.31:0/737538572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f019d750 con 0x7f79f0101de0 2024-09-16T00:41:15.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79e77fe640 1 -- 172.21.15.31:0/737538572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ec007d50 con 0x7f79f0101de0 2024-09-16T00:41:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79e77fe640 1 -- 172.21.15.31:0/737538572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00f450 con 0x7f79f0101de0 2024-09-16T00:41:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.570+0000 7f79f84f9640 1 -- 172.21.15.31:0/737538572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01a5030 con 0x7f79f0101de0 2024-09-16T00:41:15.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.573+0000 7f79fa8d2740 1 Processor -- start 2024-09-16T00:41:15.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.573+0000 7f79e77fe640 1 -- 172.21.15.31:0/737538572 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f79ec005020 con 0x7f79f0101de0 2024-09-16T00:41:15.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79fa8d2740 1 -- start start 2024-09-16T00:41:15.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad34a0 0x561354ad3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354ade140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ade680 0x561354ae0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561354a392e0 con 0x561354ad34a0 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5613549e0ce0 con 0x561354ad3dc0 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561354a352a0 con 0x561354ade680 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79f6a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad34a0 0x561354ad3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79f6a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad34a0 0x561354ad3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60458/0 (socket says 172.21.15.31:60458) 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79f6a96640 1 -- 172.21.15.31:0/2222775351 learned_addr learned my addr 172.21.15.31:0/2222775351 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79f6295640 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354ade140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.574+0000 7f79f5a94640 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ade680 0x561354ae0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79f6a96640 1 -- 172.21.15.31:0/2222775351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ade680 msgr2=0x561354ae0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:15.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79f6a96640 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ade680 0x561354ae0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79f6a96640 1 -- 172.21.15.31:0/2222775351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 msgr2=0x561354ade140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79f6a96640 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354ade140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79f6a96640 1 -- 172.21.15.31:0/2222775351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613549c5440 con 0x561354ad34a0 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79f5a94640 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ade680 0x561354ae0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79f6295640 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354ade140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79f6a96640 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad34a0 0x561354ad3880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009f70 tx=0x7f79e8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4c999b4cbf5614 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79e77fe640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f79dc0a3240 0x7f79dc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79e4ff9640 1 -- 172.21.15.31:0/2222775351 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8016070 con 0x561354ad34a0 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79e77fe640 1 -- 172.21.15.31:0/737538572 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f79ec0cf9f0 con 0x7f79f0101de0 2024-09-16T00:41:15.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79e4ff9640 1 -- 172.21.15.31:0/2222775351 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e8004d20 con 0x561354ad34a0 2024-09-16T00:41:15.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.575+0000 7f79f6295640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f79dc0a3240 0x7f79dc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.576+0000 7f79e4ff9640 1 -- 172.21.15.31:0/2222775351 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8012950 con 0x561354ad34a0 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.576+0000 7f79f6295640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f79dc0a3240 0x7f79dc0a5700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0cd4b0 tx=0x7f79ec0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.576+0000 7f79fa8d2740 1 -- 172.21.15.31:0/2222775351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad34a0 msgr2=0x561354ad3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.576+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad34a0 0x561354ad3880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009f70 tx=0x7f79e8010800 comp rx=0 tx=0).stop 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.576+0000 7f79fa8d2740 1 -- 172.21.15.31:0/2222775351 shutdown_connections 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.576+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ade680 0x561354ae0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.576+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354ade140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.576+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/2222775351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad34a0 0x561354ad3880 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.576+0000 7f79fa8d2740 1 -- 172.21.15.31:0/2222775351 >> 172.21.15.31:0/2222775351 conn(0x561354acfa30 msgr2=0x561354acfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.577+0000 7f79fa8d2740 1 -- 172.21.15.31:0/2222775351 shutdown_connections 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.577+0000 7f79fa8d2740 1 -- 172.21.15.31:0/2222775351 wait complete. 2024-09-16T00:41:15.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.577+0000 7f79fa8d2740 1 -- start start 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.577+0000 7f79fa8d2740 1 Processor -- start 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.577+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 0x561354addf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.577+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354adb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354adb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561354ae12a0 con 0x561354ade680 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5613549f4050 con 0x561354ad3dc0 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561354ae1420 con 0x561354ad34a0 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f6a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354adb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f6a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354adb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60464/0 (socket says 172.21.15.31:60464) 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f5a94640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354adb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f6a96640 1 -- 172.21.15.31:0/4279327401 learned_addr learned my addr 172.21.15.31:0/4279327401 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f6a96640 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 0x561354addf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79fa8d2740 10 client.0 _ll_register_callbacks cb 0x561354a34160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f5a94640 1 -- 172.21.15.31:0/4279327401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 msgr2=0x561354addf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f5a94640 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 0x561354addf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f5a94640 1 -- 172.21.15.31:0/4279327401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 msgr2=0x561354adb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f5a94640 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354adb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.578+0000 7f79f5a94640 1 -- 172.21.15.31:0/4279327401 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613547aa3c0 con 0x561354ad3dc0 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.579+0000 7f79f5a94640 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354adb290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005940 tx=0x7f79e00319b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48f83a781de8180 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.579+0000 7f79d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.579+0000 7f79d9ffb640 1 -- 172.21.15.31:0/4279327401 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0035040 con 0x561354ad3dc0 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.579+0000 7f79d9ffb640 1 -- 172.21.15.31:0/4279327401 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f79e0004250 con 0x561354ad3dc0 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.579+0000 7f79fa8d2740 1 -- 172.21.15.31:0/4279327401 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e8004d20 con 0x561354ad3dc0 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79d9ffb640 1 -- 172.21.15.31:0/4279327401 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f79e0004450 con 0x561354ad3dc0 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79fa8d2740 10 client.24953 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:15.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79fa8d2740 1 -- 172.21.15.31:0/4279327401 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561354aecc80 con 0x561354ad3dc0 2024-09-16T00:41:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79d9ffb640 1 -- 172.21.15.31:0/4279327401 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e8004d20 con 0x561354ad3dc0 2024-09-16T00:41:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79d9ffb640 1 -- 172.21.15.31:0/4279327401 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f79e0004c90 con 0x561354ad3dc0 2024-09-16T00:41:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79fa8d2740 10 client.24953 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79fa8d2740 10 client.24953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79fa8d2740 20 client.24953 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561354aee0e0 0x561354af0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79fa8d2740 4 client.24953 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.580+0000 7f79fa8d2740 1 -- 172.21.15.31:0/4279327401 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561354a46850 con 0x561354aee0e0 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.581+0000 7f79f6a96640 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561354aee0e0 0x561354af0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.581+0000 7f79d9ffb640 10 client.24953 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.581+0000 7f79f6a96640 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561354aee0e0 0x561354af0510 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb314622d32602ea server_cookie=b965dfbbb8f2abb5 in_seq=0 out_seq=0 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79d9ffb640 1 -- 172.21.15.31:0/4279327401 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561354a46850 con 0x561354aee0e0 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79d9ffb640 10 client.24953 handle_command_reply: tid=0 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 client.24953 shutdown 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 -- 172.21.15.31:0/4279327401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 msgr2=0x561354adb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354adb290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005940 tx=0x7f79e00319b0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 -- 172.21.15.31:0/4279327401 shutdown_connections 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354adb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad3dc0 0x561354adb290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561354aee0e0 0x561354af0510 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/4279327401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 0x561354addf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79d9ffb640 0 client.24953 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 -- 172.21.15.31:0/4279327401 >> 172.21.15.31:0/4279327401 conn(0x561354acfa30 msgr2=0x561354aceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 -- 172.21.15.31:0/4279327401 shutdown_connections 2024-09-16T00:41:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.582+0000 7f79fa8d2740 1 -- 172.21.15.31:0/4279327401 wait complete. 2024-09-16T00:41:15.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.583+0000 7f79fa8d2740 20 client.24953 trim_cache size 0 max 16384 2024-09-16T00:41:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.603+0000 7f79fa8d2740 1 Processor -- start 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.603+0000 7f79fa8d2740 1 -- start start 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.603+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad34a0 0x561354aea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.603+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad3dc0 0x561354af5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.603+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354af6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.603+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01aaa80 con 0x561354ade680 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.603+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01ad2b0 con 0x561354ad34a0 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.603+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01ab0a0 con 0x561354ad3dc0 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354af6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354af6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60468/0 (socket says 172.21.15.31:60468) 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 -- 172.21.15.31:0/605373776 learned_addr learned my addr 172.21.15.31:0/605373776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 --2- 172.21.15.31:0/605373776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad34a0 0x561354aea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f5a94640 1 --2- 172.21.15.31:0/605373776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad3dc0 0x561354af5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 -- 172.21.15.31:0/605373776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad3dc0 msgr2=0x561354af5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 --2- 172.21.15.31:0/605373776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad3dc0 0x561354af5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 -- 172.21.15.31:0/605373776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 msgr2=0x561354af6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 --2- 172.21.15.31:0/605373776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354af6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 -- 172.21.15.31:0/605373776 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561354ada240 con 0x561354ad34a0 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79f6a96640 1 --2- 172.21.15.31:0/605373776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad34a0 0x561354aea550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79e80124c0 tx=0x7f79e80124f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39e8d488e25c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.604+0000 7f79dbfff640 1 -- 172.21.15.31:0/605373776 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8016040 con 0x561354ad34a0 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79dbfff640 1 -- 172.21.15.31:0/605373776 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e8003e50 con 0x561354ad34a0 2024-09-16T00:41:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79dbfff640 1 -- 172.21.15.31:0/605373776 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8023040 con 0x561354ad34a0 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79fa8d2740 1 -- 172.21.15.31:0/605373776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad34a0 msgr2=0x561354aea550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/605373776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad34a0 0x561354aea550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79e80124c0 tx=0x7f79e80124f0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79fa8d2740 1 -- 172.21.15.31:0/605373776 shutdown_connections 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/605373776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ade680 0x561354af6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/605373776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad3dc0 0x561354af5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/605373776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ad34a0 0x561354aea550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79fa8d2740 1 -- 172.21.15.31:0/605373776 >> 172.21.15.31:0/605373776 conn(0x561354acfa30 msgr2=0x561354ad68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79fa8d2740 1 -- 172.21.15.31:0/605373776 shutdown_connections 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.605+0000 7f79fa8d2740 1 -- 172.21.15.31:0/605373776 wait complete. 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79fa8d2740 1 -- start start 2024-09-16T00:41:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79fa8d2740 1 Processor -- start 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 0x561354ae23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad3dc0 0x561354ae2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79fa8d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ade680 0x561354ae2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01ab770 con 0x561354ad3dc0 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01acbe0 con 0x561354ade680 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79fa8d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01abe40 con 0x561354ad34a0 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79fa8d2740 10 client.0 _ll_register_callbacks cb 0x7f79e0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79f5a94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad3dc0 0x561354ae2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79f5a94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad3dc0 0x561354ae2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60478/0 (socket says 172.21.15.31:60478) 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79f5a94640 1 -- 172.21.15.31:0/1613287066 learned_addr learned my addr 172.21.15.31:0/1613287066 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79f6a96640 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 0x561354ae23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.606+0000 7f79f6a96640 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ade680 0x561354ae2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79f5a94640 1 -- 172.21.15.31:0/1613287066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 msgr2=0x561354ae23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79f5a94640 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 0x561354ae23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79f5a94640 1 -- 172.21.15.31:0/1613287066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ade680 msgr2=0x561354ae2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79f5a94640 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ade680 0x561354ae2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79f5a94640 1 -- 172.21.15.31:0/1613287066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561354b57db0 con 0x561354ad3dc0 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79f6a96640 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 0x561354ae23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79f5a94640 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad3dc0 0x561354ae2910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x561354adc840 tx=0x7f79e0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6135e295ac87abac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79d9ffb640 1 -- 172.21.15.31:0/1613287066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0035040 con 0x561354ad3dc0 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79d9ffb640 1 -- 172.21.15.31:0/1613287066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f79e0043040 con 0x561354ad3dc0 2024-09-16T00:41:15.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79fa8d2740 1 -- 172.21.15.31:0/1613287066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e8003e50 con 0x561354ad3dc0 2024-09-16T00:41:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79d9ffb640 1 -- 172.21.15.31:0/1613287066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f79e00315b0 con 0x561354ad3dc0 2024-09-16T00:41:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79fa8d2740 10 client.15243 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.607+0000 7f79fa8d2740 1 -- 172.21.15.31:0/1613287066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561354b59080 con 0x561354ad3dc0 2024-09-16T00:41:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.608+0000 7f79d9ffb640 1 -- 172.21.15.31:0/1613287066 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e8003e50 con 0x561354ad3dc0 2024-09-16T00:41:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.608+0000 7f79d9ffb640 1 -- 172.21.15.31:0/1613287066 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f79e0004ca0 con 0x561354ad3dc0 2024-09-16T00:41:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.608+0000 7f79fa8d2740 10 client.15243 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.608+0000 7f79fa8d2740 10 client.15243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.608+0000 7f79fa8d2740 20 client.15243 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.608+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561354b59830 0x561354b5bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:15.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.608+0000 7f79fa8d2740 4 client.15243 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:41:15.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.608+0000 7f79fa8d2740 1 -- 172.21.15.31:0/1613287066 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561354a40d60 con 0x561354b59830 2024-09-16T00:41:15.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.608+0000 7f79f6a96640 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561354b59830 0x561354b5bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:15.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79f6a96640 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561354b59830 0x561354b5bc50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d6c4101e6ae37e server_cookie=18061b9191dfc6da in_seq=0 out_seq=0 2024-09-16T00:41:15.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79d9ffb640 10 client.15243 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:15.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79d9ffb640 1 -- 172.21.15.31:0/1613287066 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561354a40d60 con 0x561354b59830 2024-09-16T00:41:15.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79d9ffb640 10 client.15243 handle_command_reply: tid=0 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 client.15243 shutdown 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 -- 172.21.15.31:0/1613287066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad3dc0 msgr2=0x561354ae2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad3dc0 0x561354ae2910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x561354adc840 tx=0x7f79e0005270 comp rx=0 tx=0).stop 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 -- 172.21.15.31:0/1613287066 shutdown_connections 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561354ade680 0x561354ae2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561354ad3dc0 0x561354ae2910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561354b59830 0x561354b5bc50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 --2- 172.21.15.31:0/1613287066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561354ad34a0 0x561354ae23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 -- 172.21.15.31:0/1613287066 >> 172.21.15.31:0/1613287066 conn(0x561354acfa30 msgr2=0x561354ad4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79d9ffb640 0 client.15243 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 -- 172.21.15.31:0/1613287066 shutdown_connections 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.609+0000 7f79fa8d2740 1 -- 172.21.15.31:0/1613287066 wait complete. 2024-09-16T00:41:15.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79fa8d2740 20 client.15243 trim_cache size 0 max 16384 2024-09-16T00:41:15.616 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:15.616 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:41:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:41:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:41:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "6145836c-8f4e-4764-acfd-a1f8962b0000": { 2024-09-16T00:41:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:41:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "6145836c-8f4e-4764-acfd-a1f8962b0000", 2024-09-16T00:41:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:41:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:41:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:41:15.617 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:41:15.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 -- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f79dc0a3240 msgr2=0x7f79dc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:15.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f79dc0a3240 0x7f79dc0a5700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0cd4b0 tx=0x7f79ec0d5530 comp rx=0 tx=0).stop 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 -- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0101de0 msgr2=0x7f79f01984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0101de0 0x7f79f01984b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c970 tx=0x7f79ec00ce30 comp rx=0 tx=0).stop 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 -- 172.21.15.31:0/737538572 shutdown_connections 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79f0106f10 0x7f79f019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f79dc0a3240 0x7f79dc0a5700 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f0102790 0x7f79f01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 --2- 172.21.15.31:0/737538572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f0101de0 0x7f79f01984b0 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c970 tx=0x7f79ec00ce30 comp rx=0 tx=0).stop 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.610+0000 7f79f84f9640 1 -- 172.21.15.31:0/737538572 >> 172.21.15.31:0/737538572 conn(0x7f79f00fd8c0 msgr2=0x7f79f00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.611+0000 7f79f84f9640 1 -- 172.21.15.31:0/737538572 shutdown_connections 2024-09-16T00:41:15.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:15.611+0000 7f79f84f9640 1 -- 172.21.15.31:0/737538572 wait complete. 2024-09-16T00:41:16.294 INFO:tasks.cephfs.filesystem:scrub status for tag:6145836c-8f4e-4764-acfd-a1f8962b0000 - {'path': '/', 'tag': '6145836c-8f4e-4764-acfd-a1f8962b0000', 'options': 'recursive,force'} 2024-09-16T00:41:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.818+0000 7f8d30b80640 1 -- 172.21.15.31:0/4001777388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 msgr2=0x7f8d2c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.818+0000 7f8d30b80640 1 --2- 172.21.15.31:0/4001777388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 0x7f8d2c107b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009930 tx=0x7f8d1402a420 comp rx=0 tx=0).stop 2024-09-16T00:41:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.818+0000 7f8d30b80640 1 -- 172.21.15.31:0/4001777388 shutdown_connections 2024-09-16T00:41:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.818+0000 7f8d30b80640 1 --2- 172.21.15.31:0/4001777388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d2c1082a0 0x7f8d2c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:16.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.818+0000 7f8d30b80640 1 --2- 172.21.15.31:0/4001777388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 0x7f8d2c107b90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:16.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.818+0000 7f8d30b80640 1 --2- 172.21.15.31:0/4001777388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d2c102d30 0x7f8d2c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:16.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.818+0000 7f8d30b80640 1 -- 172.21.15.31:0/4001777388 >> 172.21.15.31:0/4001777388 conn(0x7f8d2c0fc7d0 msgr2=0x7f8d2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:16.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.818+0000 7f8d30b80640 1 -- 172.21.15.31:0/4001777388 shutdown_connections 2024-09-16T00:41:16.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.818+0000 7f8d30b80640 1 -- 172.21.15.31:0/4001777388 wait complete. 2024-09-16T00:41:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d30b80640 1 Processor -- start 2024-09-16T00:41:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d30b80640 1 -- start start 2024-09-16T00:41:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d30b80640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d2c102d30 0x7f8d2c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d30b80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 0x7f8d2c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d30b80640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d2c1082a0 0x7f8d2c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d30b80640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1120c0 con 0x7f8d2c1036e0 2024-09-16T00:41:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d30b80640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c111f40 con 0x7f8d2c1082a0 2024-09-16T00:41:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d30b80640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c112240 con 0x7f8d2c102d30 2024-09-16T00:41:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d23fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 0x7f8d2c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d23fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 0x7f8d2c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60482/0 (socket says 172.21.15.31:60482) 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d23fff640 1 -- 172.21.15.31:0/2745469072 learned_addr learned my addr 172.21.15.31:0/2745469072 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d2ad9d640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d2c1082a0 0x7f8d2c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d2a59c640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d2c102d30 0x7f8d2c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d23fff640 1 -- 172.21.15.31:0/2745469072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d2c102d30 msgr2=0x7f8d2c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d23fff640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d2c102d30 0x7f8d2c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d23fff640 1 -- 172.21.15.31:0/2745469072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d2c1082a0 msgr2=0x7f8d2c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d23fff640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d2c1082a0 0x7f8d2c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d23fff640 1 -- 172.21.15.31:0/2745469072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c1a6b00 con 0x7f8d2c1036e0 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.819+0000 7f8d2ad9d640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d2c1082a0 0x7f8d2c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.820+0000 7f8d23fff640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 0x7f8d2c100ed0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8d14005940 tx=0x7f8d140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.820+0000 7f8d237fe640 1 -- 172.21.15.31:0/2745469072 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1403f030 con 0x7f8d2c1036e0 2024-09-16T00:41:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.820+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c1a6d90 con 0x7f8d2c1036e0 2024-09-16T00:41:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.820+0000 7f8d237fe640 1 -- 172.21.15.31:0/2745469072 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d14003ce0 con 0x7f8d2c1036e0 2024-09-16T00:41:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.820+0000 7f8d237fe640 1 -- 172.21.15.31:0/2745469072 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14035070 con 0x7f8d2c1036e0 2024-09-16T00:41:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.820+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c1a7210 con 0x7f8d2c1036e0 2024-09-16T00:41:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.821+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d2c103110 con 0x7f8d2c1036e0 2024-09-16T00:41:16.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.822+0000 7f8d237fe640 1 -- 172.21.15.31:0/2745469072 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8d14042030 con 0x7f8d2c1036e0 2024-09-16T00:41:16.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.823+0000 7f8d237fe640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8d180a3240 0x7f8d180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:16.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.823+0000 7f8d2a59c640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8d180a3240 0x7f8d180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:16.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.823+0000 7f8d237fe640 1 -- 172.21.15.31:0/2745469072 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f8d140f01f0 con 0x7f8d2c1036e0 2024-09-16T00:41:16.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.823+0000 7f8d2a59c640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8d180a3240 0x7f8d180a5700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009890 tx=0x7f8d1c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:16.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:16.827+0000 7f8d237fe640 1 -- 172.21.15.31:0/2745469072 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d140bcb60 con 0x7f8d2c1036e0 2024-09-16T00:41:16.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:16 smithi031 ceph-mon[30956]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-16T00:41:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.004+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d2c062630 con 0x7f8d180a3240 2024-09-16T00:41:17.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.012+0000 7f8d237fe640 1 -- 172.21.15.31:0/2745469072 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340409 (secure 0 0 0) 0x7f8d2c062630 con 0x7f8d180a3240 2024-09-16T00:41:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:41:17.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8d180a3240 msgr2=0x7f8d180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:17.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8d180a3240 0x7f8d180a5700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009890 tx=0x7f8d1c008040 comp rx=0 tx=0).stop 2024-09-16T00:41:17.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 msgr2=0x7f8d2c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:17.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 0x7f8d2c100ed0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8d14005940 tx=0x7f8d140042f0 comp rx=0 tx=0).stop 2024-09-16T00:41:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 shutdown_connections 2024-09-16T00:41:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d2c1082a0 0x7f8d2c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d2c1036e0 0x7f8d2c100ed0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8d180a3240 0x7f8d180a5700 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 --2- 172.21.15.31:0/2745469072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d2c102d30 0x7f8d2c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 >> 172.21.15.31:0/2745469072 conn(0x7f8d2c0fc7d0 msgr2=0x7f8d2c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 shutdown_connections 2024-09-16T00:41:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:17.016+0000 7f8d30b80640 1 -- 172.21.15.31:0/2745469072 wait complete. 2024-09-16T00:41:17.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:16 smithi102 ceph-mon[35364]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-16T00:41:17.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:16 smithi081 ceph-mon[35539]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-16T00:41:17.698 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-16T00:41:15.688303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660841,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":348,"num_read_kb":292,"num_write":277,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":717,"ondisk_log_size":717,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400908,"kb_used_data":14156,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540916,"statfs":{"total":1151940427776,"available":1151529897984,"internally_reserved":0,"allocated":14495744,"data_stored":8185985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003509"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586194+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:40:59.586194+0000","last_peered":"2024-09-16T00:40:59.586194+0000","last_clean":"2024-09-16T00:40:59.586194+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:40:59.586194+0000","last_undegraded":"2024-09-16T00:40:59.586194+0000","last_fullsized":"2024-09-16T00:40:59.586194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589958+0000","last_change":"2024-09-16T00:40:59.589958+0000","last_active":"2024-09-16T00:40:59.589958+0000","last_peered":"2024-09-16T00:40:59.589958+0000","last_clean":"2024-09-16T00:40:59.589958+0000","last_became_active":"2024-09-16T00:40:59.589549+0000","last_became_peered":"2024-09-16T00:40:59.589549+0000","last_unstale":"2024-09-16T00:40:59.589958+0000","last_undegraded":"2024-09-16T00:40:59.589958+0000","last_fullsized":"2024-09-16T00:40:59.589958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:33.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588133+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:40:59.588133+0000","last_peered":"2024-09-16T00:40:59.588133+0000","last_clean":"2024-09-16T00:40:59.588133+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:40:59.588133+0000","last_undegraded":"2024-09-16T00:40:59.588133+0000","last_fullsized":"2024-09-16T00:40:59.588133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747727+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:40:59.747727+0000","last_peered":"2024-09-16T00:40:59.747727+0000","last_clean":"2024-09-16T00:40:59.747727+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:40:59.747727+0000","last_undegraded":"2024-09-16T00:40:59.747727+0000","last_fullsized":"2024-09-16T00:40:59.747727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162291+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:41:00.162291+0000","last_peered":"2024-09-16T00:41:00.162291+0000","last_clean":"2024-09-16T00:41:00.162291+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:41:00.162291+0000","last_undegraded":"2024-09-16T00:41:00.162291+0000","last_fullsized":"2024-09-16T00:41:00.162291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589935+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:40:59.589935+0000","last_peered":"2024-09-16T00:40:59.589935+0000","last_clean":"2024-09-16T00:40:59.589935+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:40:59.589935+0000","last_undegraded":"2024-09-16T00:40:59.589935+0000","last_fullsized":"2024-09-16T00:40:59.589935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586994+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:40:59.586994+0000","last_peered":"2024-09-16T00:40:59.586994+0000","last_clean":"2024-09-16T00:40:59.586994+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:40:59.586994+0000","last_undegraded":"2024-09-16T00:40:59.586994+0000","last_fullsized":"2024-09-16T00:40:59.586994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587478+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:40:59.587478+0000","last_peered":"2024-09-16T00:40:59.587478+0000","last_clean":"2024-09-16T00:40:59.587478+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:40:59.587478+0000","last_undegraded":"2024-09-16T00:40:59.587478+0000","last_fullsized":"2024-09-16T00:40:59.587478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590172+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:40:59.590172+0000","last_peered":"2024-09-16T00:40:59.590172+0000","last_clean":"2024-09-16T00:40:59.590172+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:40:59.590172+0000","last_undegraded":"2024-09-16T00:40:59.590172+0000","last_fullsized":"2024-09-16T00:40:59.590172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587026+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:40:59.587026+0000","last_peered":"2024-09-16T00:40:59.587026+0000","last_clean":"2024-09-16T00:40:59.587026+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:40:59.587026+0000","last_undegraded":"2024-09-16T00:40:59.587026+0000","last_fullsized":"2024-09-16T00:40:59.587026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.596631+0000","last_change":"2024-09-16T00:40:59.596631+0000","last_active":"2024-09-16T00:40:59.596631+0000","last_peered":"2024-09-16T00:40:59.596631+0000","last_clean":"2024-09-16T00:40:59.596631+0000","last_became_active":"2024-09-16T00:40:59.596165+0000","last_became_peered":"2024-09-16T00:40:59.596165+0000","last_unstale":"2024-09-16T00:40:59.596631+0000","last_undegraded":"2024-09-16T00:40:59.596631+0000","last_fullsized":"2024-09-16T00:40:59.596631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:56.918291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,1],"acting":[9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590072+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:40:59.590072+0000","last_peered":"2024-09-16T00:40:59.590072+0000","last_clean":"2024-09-16T00:40:59.590072+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:40:59.590072+0000","last_undegraded":"2024-09-16T00:40:59.590072+0000","last_fullsized":"2024-09-16T00:40:59.590072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592530+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:40:59.592530+0000","last_peered":"2024-09-16T00:40:59.592530+0000","last_clean":"2024-09-16T00:40:59.592530+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:40:59.592530+0000","last_undegraded":"2024-09-16T00:40:59.592530+0000","last_fullsized":"2024-09-16T00:40:59.592530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586270+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:40:59.586270+0000","last_peered":"2024-09-16T00:40:59.586270+0000","last_clean":"2024-09-16T00:40:59.586270+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:40:59.586270+0000","last_undegraded":"2024-09-16T00:40:59.586270+0000","last_fullsized":"2024-09-16T00:40:59.586270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590233+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:40:59.590233+0000","last_peered":"2024-09-16T00:40:59.590233+0000","last_clean":"2024-09-16T00:40:59.590233+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:40:59.590233+0000","last_undegraded":"2024-09-16T00:40:59.590233+0000","last_fullsized":"2024-09-16T00:40:59.590233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747373+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:40:59.747373+0000","last_peered":"2024-09-16T00:40:59.747373+0000","last_clean":"2024-09-16T00:40:59.747373+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:40:59.747373+0000","last_undegraded":"2024-09-16T00:40:59.747373+0000","last_fullsized":"2024-09-16T00:40:59.747373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747467+0000","last_change":"2024-09-16T00:39:59.591063+0000","last_active":"2024-09-16T00:40:59.747467+0000","last_peered":"2024-09-16T00:40:59.747467+0000","last_clean":"2024-09-16T00:40:59.747467+0000","last_became_active":"2024-09-16T00:39:59.590574+0000","last_became_peered":"2024-09-16T00:39:59.590574+0000","last_unstale":"2024-09-16T00:40:59.747467+0000","last_undegraded":"2024-09-16T00:40:59.747467+0000","last_fullsized":"2024-09-16T00:40:59.747467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:57.020592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590255+0000","last_change":"2024-09-16T00:39:59.595772+0000","last_active":"2024-09-16T00:40:59.590255+0000","last_peered":"2024-09-16T00:40:59.590255+0000","last_clean":"2024-09-16T00:40:59.590255+0000","last_became_active":"2024-09-16T00:39:59.590772+0000","last_became_peered":"2024-09-16T00:39:59.590772+0000","last_unstale":"2024-09-16T00:40:59.590255+0000","last_undegraded":"2024-09-16T00:40:59.590255+0000","last_fullsized":"2024-09-16T00:40:59.590255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:53.236019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747573+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:40:59.747573+0000","last_peered":"2024-09-16T00:40:59.747573+0000","last_clean":"2024-09-16T00:40:59.747573+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:40:59.747573+0000","last_undegraded":"2024-09-16T00:40:59.747573+0000","last_fullsized":"2024-09-16T00:40:59.747573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747545+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:40:59.747545+0000","last_peered":"2024-09-16T00:40:59.747545+0000","last_clean":"2024-09-16T00:40:59.747545+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:40:59.747545+0000","last_undegraded":"2024-09-16T00:40:59.747545+0000","last_fullsized":"2024-09-16T00:40:59.747545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585893+0000","last_change":"2024-09-16T00:39:59.386827+0000","last_active":"2024-09-16T00:40:59.585893+0000","last_peered":"2024-09-16T00:40:59.585893+0000","last_clean":"2024-09-16T00:40:59.585893+0000","last_became_active":"2024-09-16T00:39:59.386389+0000","last_became_peered":"2024-09-16T00:39:59.386389+0000","last_unstale":"2024-09-16T00:40:59.585893+0000","last_undegraded":"2024-09-16T00:40:59.585893+0000","last_fullsized":"2024-09-16T00:40:59.585893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:12:21.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585940+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:40:59.585940+0000","last_peered":"2024-09-16T00:40:59.585940+0000","last_clean":"2024-09-16T00:40:59.585940+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:40:59.585940+0000","last_undegraded":"2024-09-16T00:40:59.585940+0000","last_fullsized":"2024-09-16T00:40:59.585940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587363+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:40:59.587363+0000","last_peered":"2024-09-16T00:40:59.587363+0000","last_clean":"2024-09-16T00:40:59.587363+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:40:59.587363+0000","last_undegraded":"2024-09-16T00:40:59.587363+0000","last_fullsized":"2024-09-16T00:40:59.587363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590175+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:40:59.590175+0000","last_peered":"2024-09-16T00:40:59.590175+0000","last_clean":"2024-09-16T00:40:59.590175+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:40:59.590175+0000","last_undegraded":"2024-09-16T00:40:59.590175+0000","last_fullsized":"2024-09-16T00:40:59.590175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162114+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:41:00.162114+0000","last_peered":"2024-09-16T00:41:00.162114+0000","last_clean":"2024-09-16T00:41:00.162114+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:41:00.162114+0000","last_undegraded":"2024-09-16T00:41:00.162114+0000","last_fullsized":"2024-09-16T00:41:00.162114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590740+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:40:59.590740+0000","last_peered":"2024-09-16T00:40:59.590740+0000","last_clean":"2024-09-16T00:40:59.590740+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:40:59.590740+0000","last_undegraded":"2024-09-16T00:40:59.590740+0000","last_fullsized":"2024-09-16T00:40:59.590740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588415+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:40:59.588415+0000","last_peered":"2024-09-16T00:40:59.588415+0000","last_clean":"2024-09-16T00:40:59.588415+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:40:59.588415+0000","last_undegraded":"2024-09-16T00:40:59.588415+0000","last_fullsized":"2024-09-16T00:40:59.588415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747653+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:40:59.747653+0000","last_peered":"2024-09-16T00:40:59.747653+0000","last_clean":"2024-09-16T00:40:59.747653+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:40:59.747653+0000","last_undegraded":"2024-09-16T00:40:59.747653+0000","last_fullsized":"2024-09-16T00:40:59.747653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:04.350207+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:41:04.350207+0000","last_peered":"2024-09-16T00:41:04.350207+0000","last_clean":"2024-09-16T00:41:04.350207+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:41:04.350207+0000","last_undegraded":"2024-09-16T00:41:04.350207+0000","last_fullsized":"2024-09-16T00:41:04.350207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747093+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:40:59.747093+0000","last_peered":"2024-09-16T00:40:59.747093+0000","last_clean":"2024-09-16T00:40:59.747093+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:40:59.747093+0000","last_undegraded":"2024-09-16T00:40:59.747093+0000","last_fullsized":"2024-09-16T00:40:59.747093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595527+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:40:59.595527+0000","last_peered":"2024-09-16T00:40:59.595527+0000","last_clean":"2024-09-16T00:40:59.595527+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:40:59.595527+0000","last_undegraded":"2024-09-16T00:40:59.595527+0000","last_fullsized":"2024-09-16T00:40:59.595527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587025+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:40:59.587025+0000","last_peered":"2024-09-16T00:40:59.587025+0000","last_clean":"2024-09-16T00:40:59.587025+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:40:59.587025+0000","last_undegraded":"2024-09-16T00:40:59.587025+0000","last_fullsized":"2024-09-16T00:40:59.587025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747410+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:40:59.747410+0000","last_peered":"2024-09-16T00:40:59.747410+0000","last_clean":"2024-09-16T00:40:59.747410+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:40:59.747410+0000","last_undegraded":"2024-09-16T00:40:59.747410+0000","last_fullsized":"2024-09-16T00:40:59.747410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747586+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:40:59.747586+0000","last_peered":"2024-09-16T00:40:59.747586+0000","last_clean":"2024-09-16T00:40:59.747586+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:40:59.747586+0000","last_undegraded":"2024-09-16T00:40:59.747586+0000","last_fullsized":"2024-09-16T00:40:59.747586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585296+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:40:59.585296+0000","last_peered":"2024-09-16T00:40:59.585296+0000","last_clean":"2024-09-16T00:40:59.585296+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:40:59.585296+0000","last_undegraded":"2024-09-16T00:40:59.585296+0000","last_fullsized":"2024-09-16T00:40:59.585296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586050+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:40:59.586050+0000","last_peered":"2024-09-16T00:40:59.586050+0000","last_clean":"2024-09-16T00:40:59.586050+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:40:59.586050+0000","last_undegraded":"2024-09-16T00:40:59.586050+0000","last_fullsized":"2024-09-16T00:40:59.586050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747487+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:40:59.747487+0000","last_peered":"2024-09-16T00:40:59.747487+0000","last_clean":"2024-09-16T00:40:59.747487+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:40:59.747487+0000","last_undegraded":"2024-09-16T00:40:59.747487+0000","last_fullsized":"2024-09-16T00:40:59.747487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592602+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:40:59.592602+0000","last_peered":"2024-09-16T00:40:59.592602+0000","last_clean":"2024-09-16T00:40:59.592602+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:40:59.592602+0000","last_undegraded":"2024-09-16T00:40:59.592602+0000","last_fullsized":"2024-09-16T00:40:59.592602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595268+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:40:59.595268+0000","last_peered":"2024-09-16T00:40:59.595268+0000","last_clean":"2024-09-16T00:40:59.595268+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:40:59.595268+0000","last_undegraded":"2024-09-16T00:40:59.595268+0000","last_fullsized":"2024-09-16T00:40:59.595268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.591482+0000","last_change":"2024-09-16T00:40:59.591482+0000","last_active":"2024-09-16T00:40:59.591482+0000","last_peered":"2024-09-16T00:40:59.591482+0000","last_clean":"2024-09-16T00:40:59.591482+0000","last_became_active":"2024-09-16T00:40:59.590978+0000","last_became_peered":"2024-09-16T00:40:59.590978+0000","last_unstale":"2024-09-16T00:40:59.591482+0000","last_undegraded":"2024-09-16T00:40:59.591482+0000","last_fullsized":"2024-09-16T00:40:59.591482+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:56.975117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594993+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:40:59.594993+0000","last_peered":"2024-09-16T00:40:59.594993+0000","last_clean":"2024-09-16T00:40:59.594993+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:40:59.594993+0000","last_undegraded":"2024-09-16T00:40:59.594993+0000","last_fullsized":"2024-09-16T00:40:59.594993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595391+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:40:59.595391+0000","last_peered":"2024-09-16T00:40:59.595391+0000","last_clean":"2024-09-16T00:40:59.595391+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:40:59.595391+0000","last_undegraded":"2024-09-16T00:40:59.595391+0000","last_fullsized":"2024-09-16T00:40:59.595391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594855+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:40:59.594855+0000","last_peered":"2024-09-16T00:40:59.594855+0000","last_clean":"2024-09-16T00:40:59.594855+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:40:59.594855+0000","last_undegraded":"2024-09-16T00:40:59.594855+0000","last_fullsized":"2024-09-16T00:40:59.594855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747209+0000","last_change":"2024-09-16T00:39:59.376963+0000","last_active":"2024-09-16T00:40:59.747209+0000","last_peered":"2024-09-16T00:40:59.747209+0000","last_clean":"2024-09-16T00:40:59.747209+0000","last_became_active":"2024-09-16T00:39:59.376538+0000","last_became_peered":"2024-09-16T00:39:59.376538+0000","last_unstale":"2024-09-16T00:40:59.747209+0000","last_undegraded":"2024-09-16T00:40:59.747209+0000","last_fullsized":"2024-09-16T00:40:59.747209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:02.077749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587907+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:40:59.587907+0000","last_peered":"2024-09-16T00:40:59.587907+0000","last_clean":"2024-09-16T00:40:59.587907+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:40:59.587907+0000","last_undegraded":"2024-09-16T00:40:59.587907+0000","last_fullsized":"2024-09-16T00:40:59.587907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747729+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:40:59.747729+0000","last_peered":"2024-09-16T00:40:59.747729+0000","last_clean":"2024-09-16T00:40:59.747729+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:40:59.747729+0000","last_undegraded":"2024-09-16T00:40:59.747729+0000","last_fullsized":"2024-09-16T00:40:59.747729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590591+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:40:59.590591+0000","last_peered":"2024-09-16T00:40:59.590591+0000","last_clean":"2024-09-16T00:40:59.590591+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:40:59.590591+0000","last_undegraded":"2024-09-16T00:40:59.590591+0000","last_fullsized":"2024-09-16T00:40:59.590591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592357+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:40:59.592357+0000","last_peered":"2024-09-16T00:40:59.592357+0000","last_clean":"2024-09-16T00:40:59.592357+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:40:59.592357+0000","last_undegraded":"2024-09-16T00:40:59.592357+0000","last_fullsized":"2024-09-16T00:40:59.592357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592743+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:40:59.592743+0000","last_peered":"2024-09-16T00:40:59.592743+0000","last_clean":"2024-09-16T00:40:59.592743+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:40:59.592743+0000","last_undegraded":"2024-09-16T00:40:59.592743+0000","last_fullsized":"2024-09-16T00:40:59.592743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592923+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:40:59.592923+0000","last_peered":"2024-09-16T00:40:59.592923+0000","last_clean":"2024-09-16T00:40:59.592923+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:40:59.592923+0000","last_undegraded":"2024-09-16T00:40:59.592923+0000","last_fullsized":"2024-09-16T00:40:59.592923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589988+0000","last_change":"2024-09-16T00:39:59.593442+0000","last_active":"2024-09-16T00:40:59.589988+0000","last_peered":"2024-09-16T00:40:59.589988+0000","last_clean":"2024-09-16T00:40:59.589988+0000","last_became_active":"2024-09-16T00:39:59.593039+0000","last_became_peered":"2024-09-16T00:39:59.593039+0000","last_unstale":"2024-09-16T00:40:59.589988+0000","last_undegraded":"2024-09-16T00:40:59.589988+0000","last_fullsized":"2024-09-16T00:40:59.589988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:36.644148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587371+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:40:59.587371+0000","last_peered":"2024-09-16T00:40:59.587371+0000","last_clean":"2024-09-16T00:40:59.587371+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:40:59.587371+0000","last_undegraded":"2024-09-16T00:40:59.587371+0000","last_fullsized":"2024-09-16T00:40:59.587371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592697+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:40:59.592697+0000","last_peered":"2024-09-16T00:40:59.592697+0000","last_clean":"2024-09-16T00:40:59.592697+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:40:59.592697+0000","last_undegraded":"2024-09-16T00:40:59.592697+0000","last_fullsized":"2024-09-16T00:40:59.592697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589435+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:40:59.589435+0000","last_peered":"2024-09-16T00:40:59.589435+0000","last_clean":"2024-09-16T00:40:59.589435+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:40:59.589435+0000","last_undegraded":"2024-09-16T00:40:59.589435+0000","last_fullsized":"2024-09-16T00:40:59.589435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161336+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:41:00.161336+0000","last_peered":"2024-09-16T00:41:00.161336+0000","last_clean":"2024-09-16T00:41:00.161336+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:41:00.161336+0000","last_undegraded":"2024-09-16T00:41:00.161336+0000","last_fullsized":"2024-09-16T00:41:00.161336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592993+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:40:59.592993+0000","last_peered":"2024-09-16T00:40:59.592993+0000","last_clean":"2024-09-16T00:40:59.592993+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:40:59.592993+0000","last_undegraded":"2024-09-16T00:40:59.592993+0000","last_fullsized":"2024-09-16T00:40:59.592993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595517+0000","last_change":"2024-09-16T00:39:59.594994+0000","last_active":"2024-09-16T00:40:59.595517+0000","last_peered":"2024-09-16T00:40:59.595517+0000","last_clean":"2024-09-16T00:40:59.595517+0000","last_became_active":"2024-09-16T00:39:59.590640+0000","last_became_peered":"2024-09-16T00:39:59.590640+0000","last_unstale":"2024-09-16T00:40:59.595517+0000","last_undegraded":"2024-09-16T00:40:59.595517+0000","last_fullsized":"2024-09-16T00:40:59.595517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:21.391494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587245+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:40:59.587245+0000","last_peered":"2024-09-16T00:40:59.587245+0000","last_clean":"2024-09-16T00:40:59.587245+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:40:59.587245+0000","last_undegraded":"2024-09-16T00:40:59.587245+0000","last_fullsized":"2024-09-16T00:40:59.587245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747566+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:40:59.747566+0000","last_peered":"2024-09-16T00:40:59.747566+0000","last_clean":"2024-09-16T00:40:59.747566+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:40:59.747566+0000","last_undegraded":"2024-09-16T00:40:59.747566+0000","last_fullsized":"2024-09-16T00:40:59.747566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586785+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:40:59.586785+0000","last_peered":"2024-09-16T00:40:59.586785+0000","last_clean":"2024-09-16T00:40:59.586785+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:40:59.586785+0000","last_undegraded":"2024-09-16T00:40:59.586785+0000","last_fullsized":"2024-09-16T00:40:59.586785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588117+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:40:59.588117+0000","last_peered":"2024-09-16T00:40:59.588117+0000","last_clean":"2024-09-16T00:40:59.588117+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:40:59.588117+0000","last_undegraded":"2024-09-16T00:40:59.588117+0000","last_fullsized":"2024-09-16T00:40:59.588117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588295+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:40:59.588295+0000","last_peered":"2024-09-16T00:40:59.588295+0000","last_clean":"2024-09-16T00:40:59.588295+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:40:59.588295+0000","last_undegraded":"2024-09-16T00:40:59.588295+0000","last_fullsized":"2024-09-16T00:40:59.588295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585921+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:40:59.585921+0000","last_peered":"2024-09-16T00:40:59.585921+0000","last_clean":"2024-09-16T00:40:59.585921+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:40:59.585921+0000","last_undegraded":"2024-09-16T00:40:59.585921+0000","last_fullsized":"2024-09-16T00:40:59.585921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595364+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:40:59.595364+0000","last_peered":"2024-09-16T00:40:59.595364+0000","last_clean":"2024-09-16T00:40:59.595364+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:40:59.595364+0000","last_undegraded":"2024-09-16T00:40:59.595364+0000","last_fullsized":"2024-09-16T00:40:59.595364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592245+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:40:59.592245+0000","last_peered":"2024-09-16T00:40:59.592245+0000","last_clean":"2024-09-16T00:40:59.592245+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:40:59.592245+0000","last_undegraded":"2024-09-16T00:40:59.592245+0000","last_fullsized":"2024-09-16T00:40:59.592245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590466+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:40:59.590466+0000","last_peered":"2024-09-16T00:40:59.590466+0000","last_clean":"2024-09-16T00:40:59.590466+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:40:59.590466+0000","last_undegraded":"2024-09-16T00:40:59.590466+0000","last_fullsized":"2024-09-16T00:40:59.590466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586021+0000","last_change":"2024-09-16T00:39:59.593568+0000","last_active":"2024-09-16T00:40:59.586021+0000","last_peered":"2024-09-16T00:40:59.586021+0000","last_clean":"2024-09-16T00:40:59.586021+0000","last_became_active":"2024-09-16T00:39:59.593209+0000","last_became_peered":"2024-09-16T00:39:59.593209+0000","last_unstale":"2024-09-16T00:40:59.586021+0000","last_undegraded":"2024-09-16T00:40:59.586021+0000","last_fullsized":"2024-09-16T00:40:59.586021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:56.516773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.597069+0000","last_change":"2024-09-16T00:40:59.596544+0000","last_active":"2024-09-16T00:40:59.597069+0000","last_peered":"2024-09-16T00:40:59.597069+0000","last_clean":"2024-09-16T00:40:59.597069+0000","last_became_active":"2024-09-16T00:40:59.596128+0000","last_became_peered":"2024-09-16T00:40:59.596128+0000","last_unstale":"2024-09-16T00:40:59.597069+0000","last_undegraded":"2024-09-16T00:40:59.597069+0000","last_fullsized":"2024-09-16T00:40:59.597069+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:22.691164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587694+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:40:59.587694+0000","last_peered":"2024-09-16T00:40:59.587694+0000","last_clean":"2024-09-16T00:40:59.587694+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:40:59.587694+0000","last_undegraded":"2024-09-16T00:40:59.587694+0000","last_fullsized":"2024-09-16T00:40:59.587694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747362+0000","last_change":"2024-09-16T00:39:59.380258+0000","last_active":"2024-09-16T00:40:59.747362+0000","last_peered":"2024-09-16T00:40:59.747362+0000","last_clean":"2024-09-16T00:40:59.747362+0000","last_became_active":"2024-09-16T00:39:59.379813+0000","last_became_peered":"2024-09-16T00:39:59.379813+0000","last_unstale":"2024-09-16T00:40:59.747362+0000","last_undegraded":"2024-09-16T00:40:59.747362+0000","last_fullsized":"2024-09-16T00:40:59.747362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:03.110305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595116+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:40:59.595116+0000","last_peered":"2024-09-16T00:40:59.595116+0000","last_clean":"2024-09-16T00:40:59.595116+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:40:59.595116+0000","last_undegraded":"2024-09-16T00:40:59.595116+0000","last_fullsized":"2024-09-16T00:40:59.595116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588686+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:40:59.588686+0000","last_peered":"2024-09-16T00:40:59.588686+0000","last_clean":"2024-09-16T00:40:59.588686+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:40:59.588686+0000","last_undegraded":"2024-09-16T00:40:59.588686+0000","last_fullsized":"2024-09-16T00:40:59.588686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588982+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:40:59.588982+0000","last_peered":"2024-09-16T00:40:59.588982+0000","last_clean":"2024-09-16T00:40:59.588982+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:40:59.588982+0000","last_undegraded":"2024-09-16T00:40:59.588982+0000","last_fullsized":"2024-09-16T00:40:59.588982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586881+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:40:59.586881+0000","last_peered":"2024-09-16T00:40:59.586881+0000","last_clean":"2024-09-16T00:40:59.586881+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:40:59.586881+0000","last_undegraded":"2024-09-16T00:40:59.586881+0000","last_fullsized":"2024-09-16T00:40:59.586881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":289,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:15.306590+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:41:15.306590+0000","last_peered":"2024-09-16T00:41:15.306590+0000","last_clean":"2024-09-16T00:41:15.306590+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:41:15.306590+0000","last_undegraded":"2024-09-16T00:41:15.306590+0000","last_fullsized":"2024-09-16T00:41:15.306590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69441,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":215,"num_read_kb":176,"num_write":35,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585067+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:40:59.585067+0000","last_peered":"2024-09-16T00:40:59.585067+0000","last_clean":"2024-09-16T00:40:59.585067+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:40:59.585067+0000","last_undegraded":"2024-09-16T00:40:59.585067+0000","last_fullsized":"2024-09-16T00:40:59.585067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587571+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:40:59.587571+0000","last_peered":"2024-09-16T00:40:59.587571+0000","last_clean":"2024-09-16T00:40:59.587571+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:40:59.587571+0000","last_undegraded":"2024-09-16T00:40:59.587571+0000","last_fullsized":"2024-09-16T00:40:59.587571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589244+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:40:59.589244+0000","last_peered":"2024-09-16T00:40:59.589244+0000","last_clean":"2024-09-16T00:40:59.589244+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:40:59.589244+0000","last_undegraded":"2024-09-16T00:40:59.589244+0000","last_fullsized":"2024-09-16T00:40:59.589244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590297+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:40:59.590297+0000","last_peered":"2024-09-16T00:40:59.590297+0000","last_clean":"2024-09-16T00:40:59.590297+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:40:59.590297+0000","last_undegraded":"2024-09-16T00:40:59.590297+0000","last_fullsized":"2024-09-16T00:40:59.590297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162296+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:41:00.162296+0000","last_peered":"2024-09-16T00:41:00.162296+0000","last_clean":"2024-09-16T00:41:00.162296+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:41:00.162296+0000","last_undegraded":"2024-09-16T00:41:00.162296+0000","last_fullsized":"2024-09-16T00:41:00.162296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592831+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:40:59.592831+0000","last_peered":"2024-09-16T00:40:59.592831+0000","last_clean":"2024-09-16T00:40:59.592831+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:40:59.592831+0000","last_undegraded":"2024-09-16T00:40:59.592831+0000","last_fullsized":"2024-09-16T00:40:59.592831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584869+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:40:59.584869+0000","last_peered":"2024-09-16T00:40:59.584869+0000","last_clean":"2024-09-16T00:40:59.584869+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:40:59.584869+0000","last_undegraded":"2024-09-16T00:40:59.584869+0000","last_fullsized":"2024-09-16T00:40:59.584869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587926+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:40:59.587926+0000","last_peered":"2024-09-16T00:40:59.587926+0000","last_clean":"2024-09-16T00:40:59.587926+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:40:59.587926+0000","last_undegraded":"2024-09-16T00:40:59.587926+0000","last_fullsized":"2024-09-16T00:40:59.587926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747773+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:40:59.747773+0000","last_peered":"2024-09-16T00:40:59.747773+0000","last_clean":"2024-09-16T00:40:59.747773+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:40:59.747773+0000","last_undegraded":"2024-09-16T00:40:59.747773+0000","last_fullsized":"2024-09-16T00:40:59.747773+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747139+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:40:59.747139+0000","last_peered":"2024-09-16T00:40:59.747139+0000","last_clean":"2024-09-16T00:40:59.747139+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:40:59.747139+0000","last_undegraded":"2024-09-16T00:40:59.747139+0000","last_fullsized":"2024-09-16T00:40:59.747139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161443+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:41:00.161443+0000","last_peered":"2024-09-16T00:41:00.161443+0000","last_clean":"2024-09-16T00:41:00.161443+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:41:00.161443+0000","last_undegraded":"2024-09-16T00:41:00.161443+0000","last_fullsized":"2024-09-16T00:41:00.161443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584969+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:40:59.584969+0000","last_peered":"2024-09-16T00:40:59.584969+0000","last_clean":"2024-09-16T00:40:59.584969+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:40:59.584969+0000","last_undegraded":"2024-09-16T00:40:59.584969+0000","last_fullsized":"2024-09-16T00:40:59.584969+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588578+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:40:59.588578+0000","last_peered":"2024-09-16T00:40:59.588578+0000","last_clean":"2024-09-16T00:40:59.588578+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:40:59.588578+0000","last_undegraded":"2024-09-16T00:40:59.588578+0000","last_fullsized":"2024-09-16T00:40:59.588578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747585+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:40:59.747585+0000","last_peered":"2024-09-16T00:40:59.747585+0000","last_clean":"2024-09-16T00:40:59.747585+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:40:59.747585+0000","last_undegraded":"2024-09-16T00:40:59.747585+0000","last_fullsized":"2024-09-16T00:40:59.747585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747194+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:40:59.747194+0000","last_peered":"2024-09-16T00:40:59.747194+0000","last_clean":"2024-09-16T00:40:59.747194+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:40:59.747194+0000","last_undegraded":"2024-09-16T00:40:59.747194+0000","last_fullsized":"2024-09-16T00:40:59.747194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594858+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:40:59.594858+0000","last_peered":"2024-09-16T00:40:59.594858+0000","last_clean":"2024-09-16T00:40:59.594858+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:40:59.594858+0000","last_undegraded":"2024-09-16T00:40:59.594858+0000","last_fullsized":"2024-09-16T00:40:59.594858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747376+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:40:59.747376+0000","last_peered":"2024-09-16T00:40:59.747376+0000","last_clean":"2024-09-16T00:40:59.747376+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:40:59.747376+0000","last_undegraded":"2024-09-16T00:40:59.747376+0000","last_fullsized":"2024-09-16T00:40:59.747376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588250+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:40:59.588250+0000","last_peered":"2024-09-16T00:40:59.588250+0000","last_clean":"2024-09-16T00:40:59.588250+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:40:59.588250+0000","last_undegraded":"2024-09-16T00:40:59.588250+0000","last_fullsized":"2024-09-16T00:40:59.588250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587684+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:40:59.587684+0000","last_peered":"2024-09-16T00:40:59.587684+0000","last_clean":"2024-09-16T00:40:59.587684+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:40:59.587684+0000","last_undegraded":"2024-09-16T00:40:59.587684+0000","last_fullsized":"2024-09-16T00:40:59.587684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588368+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:40:59.588368+0000","last_peered":"2024-09-16T00:40:59.588368+0000","last_clean":"2024-09-16T00:40:59.588368+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:40:59.588368+0000","last_undegraded":"2024-09-16T00:40:59.588368+0000","last_fullsized":"2024-09-16T00:40:59.588368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586131+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:40:59.586131+0000","last_peered":"2024-09-16T00:40:59.586131+0000","last_clean":"2024-09-16T00:40:59.586131+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:40:59.586131+0000","last_undegraded":"2024-09-16T00:40:59.586131+0000","last_fullsized":"2024-09-16T00:40:59.586131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588144+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:40:59.588144+0000","last_peered":"2024-09-16T00:40:59.588144+0000","last_clean":"2024-09-16T00:40:59.588144+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:40:59.588144+0000","last_undegraded":"2024-09-16T00:40:59.588144+0000","last_fullsized":"2024-09-16T00:40:59.588144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747412+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:40:59.747412+0000","last_peered":"2024-09-16T00:40:59.747412+0000","last_clean":"2024-09-16T00:40:59.747412+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:40:59.747412+0000","last_undegraded":"2024-09-16T00:40:59.747412+0000","last_fullsized":"2024-09-16T00:40:59.747412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588676+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:40:59.588676+0000","last_peered":"2024-09-16T00:40:59.588676+0000","last_clean":"2024-09-16T00:40:59.588676+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:40:59.588676+0000","last_undegraded":"2024-09-16T00:40:59.588676+0000","last_fullsized":"2024-09-16T00:40:59.588676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587600+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:40:59.587600+0000","last_peered":"2024-09-16T00:40:59.587600+0000","last_clean":"2024-09-16T00:40:59.587600+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:40:59.587600+0000","last_undegraded":"2024-09-16T00:40:59.587600+0000","last_fullsized":"2024-09-16T00:40:59.587600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162004+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:41:00.162004+0000","last_peered":"2024-09-16T00:41:00.162004+0000","last_clean":"2024-09-16T00:41:00.162004+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:41:00.162004+0000","last_undegraded":"2024-09-16T00:41:00.162004+0000","last_fullsized":"2024-09-16T00:41:00.162004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595476+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:40:59.595476+0000","last_peered":"2024-09-16T00:40:59.595476+0000","last_clean":"2024-09-16T00:40:59.595476+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:40:59.595476+0000","last_undegraded":"2024-09-16T00:40:59.595476+0000","last_fullsized":"2024-09-16T00:40:59.595476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747786+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:40:59.747786+0000","last_peered":"2024-09-16T00:40:59.747786+0000","last_clean":"2024-09-16T00:40:59.747786+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:40:59.747786+0000","last_undegraded":"2024-09-16T00:40:59.747786+0000","last_fullsized":"2024-09-16T00:40:59.747786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588960+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:40:59.588960+0000","last_peered":"2024-09-16T00:40:59.588960+0000","last_clean":"2024-09-16T00:40:59.588960+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:40:59.588960+0000","last_undegraded":"2024-09-16T00:40:59.588960+0000","last_fullsized":"2024-09-16T00:40:59.588960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.748038+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:40:59.748038+0000","last_peered":"2024-09-16T00:40:59.748038+0000","last_clean":"2024-09-16T00:40:59.748038+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:40:59.748038+0000","last_undegraded":"2024-09-16T00:40:59.748038+0000","last_fullsized":"2024-09-16T00:40:59.748038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587358+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:40:59.587358+0000","last_peered":"2024-09-16T00:40:59.587358+0000","last_clean":"2024-09-16T00:40:59.587358+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:40:59.587358+0000","last_undegraded":"2024-09-16T00:40:59.587358+0000","last_fullsized":"2024-09-16T00:40:59.587358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747301+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:40:59.747301+0000","last_peered":"2024-09-16T00:40:59.747301+0000","last_clean":"2024-09-16T00:40:59.747301+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:40:59.747301+0000","last_undegraded":"2024-09-16T00:40:59.747301+0000","last_fullsized":"2024-09-16T00:40:59.747301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587542+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:40:59.587542+0000","last_peered":"2024-09-16T00:40:59.587542+0000","last_clean":"2024-09-16T00:40:59.587542+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:40:59.587542+0000","last_undegraded":"2024-09-16T00:40:59.587542+0000","last_fullsized":"2024-09-16T00:40:59.587542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595665+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:40:59.595665+0000","last_peered":"2024-09-16T00:40:59.595665+0000","last_clean":"2024-09-16T00:40:59.595665+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:40:59.595665+0000","last_undegraded":"2024-09-16T00:40:59.595665+0000","last_fullsized":"2024-09-16T00:40:59.595665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747529+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:40:59.747529+0000","last_peered":"2024-09-16T00:40:59.747529+0000","last_clean":"2024-09-16T00:40:59.747529+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:40:59.747529+0000","last_undegraded":"2024-09-16T00:40:59.747529+0000","last_fullsized":"2024-09-16T00:40:59.747529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588319+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:40:59.588319+0000","last_peered":"2024-09-16T00:40:59.588319+0000","last_clean":"2024-09-16T00:40:59.588319+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:40:59.588319+0000","last_undegraded":"2024-09-16T00:40:59.588319+0000","last_fullsized":"2024-09-16T00:40:59.588319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586100+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:40:59.586100+0000","last_peered":"2024-09-16T00:40:59.586100+0000","last_clean":"2024-09-16T00:40:59.586100+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:40:59.586100+0000","last_undegraded":"2024-09-16T00:40:59.586100+0000","last_fullsized":"2024-09-16T00:40:59.586100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747774+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:40:59.747774+0000","last_peered":"2024-09-16T00:40:59.747774+0000","last_clean":"2024-09-16T00:40:59.747774+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:40:59.747774+0000","last_undegraded":"2024-09-16T00:40:59.747774+0000","last_fullsized":"2024-09-16T00:40:59.747774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587142+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:40:59.587142+0000","last_peered":"2024-09-16T00:40:59.587142+0000","last_clean":"2024-09-16T00:40:59.587142+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:40:59.587142+0000","last_undegraded":"2024-09-16T00:40:59.587142+0000","last_fullsized":"2024-09-16T00:40:59.587142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161683+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:41:00.161683+0000","last_peered":"2024-09-16T00:41:00.161683+0000","last_clean":"2024-09-16T00:41:00.161683+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:41:00.161683+0000","last_undegraded":"2024-09-16T00:41:00.161683+0000","last_fullsized":"2024-09-16T00:41:00.161683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161806+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:41:00.161806+0000","last_peered":"2024-09-16T00:41:00.161806+0000","last_clean":"2024-09-16T00:41:00.161806+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:41:00.161806+0000","last_undegraded":"2024-09-16T00:41:00.161806+0000","last_fullsized":"2024-09-16T00:41:00.161806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595267+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:40:59.595267+0000","last_peered":"2024-09-16T00:40:59.595267+0000","last_clean":"2024-09-16T00:40:59.595267+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:40:59.595267+0000","last_undegraded":"2024-09-16T00:40:59.595267+0000","last_fullsized":"2024-09-16T00:40:59.595267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590469+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:40:59.590469+0000","last_peered":"2024-09-16T00:40:59.590469+0000","last_clean":"2024-09-16T00:40:59.590469+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:40:59.590469+0000","last_undegraded":"2024-09-16T00:40:59.590469+0000","last_fullsized":"2024-09-16T00:40:59.590469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592840+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:40:59.592840+0000","last_peered":"2024-09-16T00:40:59.592840+0000","last_clean":"2024-09-16T00:40:59.592840+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:40:59.592840+0000","last_undegraded":"2024-09-16T00:40:59.592840+0000","last_fullsized":"2024-09-16T00:40:59.592840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747793+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:40:59.747793+0000","last_peered":"2024-09-16T00:40:59.747793+0000","last_clean":"2024-09-16T00:40:59.747793+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:40:59.747793+0000","last_undegraded":"2024-09-16T00:40:59.747793+0000","last_fullsized":"2024-09-16T00:40:59.747793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587933+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:40:59.587933+0000","last_peered":"2024-09-16T00:40:59.587933+0000","last_clean":"2024-09-16T00:40:59.587933+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:40:59.587933+0000","last_undegraded":"2024-09-16T00:40:59.587933+0000","last_fullsized":"2024-09-16T00:40:59.587933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747138+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:40:59.747138+0000","last_peered":"2024-09-16T00:40:59.747138+0000","last_clean":"2024-09-16T00:40:59.747138+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:40:59.747138+0000","last_undegraded":"2024-09-16T00:40:59.747138+0000","last_fullsized":"2024-09-16T00:40:59.747138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592328+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:40:59.592328+0000","last_peered":"2024-09-16T00:40:59.592328+0000","last_clean":"2024-09-16T00:40:59.592328+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:40:59.592328+0000","last_undegraded":"2024-09-16T00:40:59.592328+0000","last_fullsized":"2024-09-16T00:40:59.592328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747364+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:40:59.747364+0000","last_peered":"2024-09-16T00:40:59.747364+0000","last_clean":"2024-09-16T00:40:59.747364+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:40:59.747364+0000","last_undegraded":"2024-09-16T00:40:59.747364+0000","last_fullsized":"2024-09-16T00:40:59.747364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586035+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:40:59.586035+0000","last_peered":"2024-09-16T00:40:59.586035+0000","last_clean":"2024-09-16T00:40:59.586035+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:40:59.586035+0000","last_undegraded":"2024-09-16T00:40:59.586035+0000","last_fullsized":"2024-09-16T00:40:59.586035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590395+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:40:59.590395+0000","last_peered":"2024-09-16T00:40:59.590395+0000","last_clean":"2024-09-16T00:40:59.590395+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:40:59.590395+0000","last_undegraded":"2024-09-16T00:40:59.590395+0000","last_fullsized":"2024-09-16T00:40:59.590395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162106+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:41:00.162106+0000","last_peered":"2024-09-16T00:41:00.162106+0000","last_clean":"2024-09-16T00:41:00.162106+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:41:00.162106+0000","last_undegraded":"2024-09-16T00:41:00.162106+0000","last_fullsized":"2024-09-16T00:41:00.162106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162139+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:41:00.162139+0000","last_peered":"2024-09-16T00:41:00.162139+0000","last_clean":"2024-09-16T00:41:00.162139+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:41:00.162139+0000","last_undegraded":"2024-09-16T00:41:00.162139+0000","last_fullsized":"2024-09-16T00:41:00.162139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747779+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:40:59.747779+0000","last_peered":"2024-09-16T00:40:59.747779+0000","last_clean":"2024-09-16T00:40:59.747779+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:40:59.747779+0000","last_undegraded":"2024-09-16T00:40:59.747779+0000","last_fullsized":"2024-09-16T00:40:59.747779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70239,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":222,"num_read_kb":183,"num_write":94,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":210717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547242,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":518379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1720320,"data_stored":1178382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939700,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":518961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168634,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":518983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1638400,"data_stored":1113425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":514283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1642496,"data_stored":1108929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":518401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":514387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444121,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":583758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":583724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":514373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:41:17.703 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:41:17.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:41:17.968 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:18.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:17 smithi102 ceph-mon[35364]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:18.089 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:17 smithi031 ceph-mon[30956]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:18.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:17 smithi081 ceph-mon[35539]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.501+0000 7f8acbfac640 1 -- 172.21.15.31:0/694942030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 msgr2=0x7f8ac410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.501+0000 7f8acbfac640 1 --2- 172.21.15.31:0/694942030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 0x7f8ac410ea90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009ca0 tx=0x7f8abc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.501+0000 7f8acbfac640 1 -- 172.21.15.31:0/694942030 shutdown_connections 2024-09-16T00:41:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.501+0000 7f8acbfac640 1 --2- 172.21.15.31:0/694942030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 0x7f8ac410ea90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.501+0000 7f8acbfac640 1 --2- 172.21.15.31:0/694942030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ac4102790 0x7f8ac41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.501+0000 7f8acbfac640 1 --2- 172.21.15.31:0/694942030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ac4101de0 0x7f8ac41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.501+0000 7f8acbfac640 1 -- 172.21.15.31:0/694942030 >> 172.21.15.31:0/694942030 conn(0x7f8ac40fd8c0 msgr2=0x7f8ac40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.501+0000 7f8acbfac640 1 -- 172.21.15.31:0/694942030 shutdown_connections 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.501+0000 7f8acbfac640 1 -- 172.21.15.31:0/694942030 wait complete. 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8acbfac640 1 Processor -- start 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8acbfac640 1 -- start start 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8acbfac640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ac4101de0 0x7f8ac419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8acbfac640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ac4102790 0x7f8ac419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8acbfac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 0x7f8ac419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8acbfac640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4111f70 con 0x7f8ac4106f10 2024-09-16T00:41:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8acbfac640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4111df0 con 0x7f8ac4102790 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8acbfac640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41120f0 con 0x7f8ac4101de0 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8aca549640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 0x7f8ac419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8aca549640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 0x7f8ac419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49260/0 (socket says 172.21.15.31:49260) 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8aca549640 1 -- 172.21.15.31:0/2043091286 learned_addr learned my addr 172.21.15.31:0/2043091286 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8ac9d48640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ac4101de0 0x7f8ac419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.502+0000 7f8ac9547640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ac4102790 0x7f8ac419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8aca549640 1 -- 172.21.15.31:0/2043091286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ac4101de0 msgr2=0x7f8ac419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8aca549640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ac4101de0 0x7f8ac419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8aca549640 1 -- 172.21.15.31:0/2043091286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ac4102790 msgr2=0x7f8ac419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8aca549640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ac4102790 0x7f8ac419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8aca549640 1 -- 172.21.15.31:0/2043091286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac419f580 con 0x7f8ac4106f10 2024-09-16T00:41:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8ac9547640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ac4102790 0x7f8ac419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8aca549640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 0x7f8ac419eea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8abc031a20 tx=0x7f8abc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8abaffd640 1 -- 172.21.15.31:0/2043091286 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc003b70 con 0x7f8ac4106f10 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8abaffd640 1 -- 172.21.15.31:0/2043091286 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8abc003d10 con 0x7f8ac4106f10 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac419f810 con 0x7f8ac4106f10 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8abaffd640 1 -- 172.21.15.31:0/2043091286 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc02fc40 con 0x7f8ac4106f10 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.503+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac41a7150 con 0x7f8ac4106f10 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.504+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac41021c0 con 0x7f8ac4106f10 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.505+0000 7f8abaffd640 1 -- 172.21.15.31:0/2043091286 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8abc004050 con 0x7f8ac4106f10 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.506+0000 7f8abaffd640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ab00a3240 0x7f8ab00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.506+0000 7f8ac9d48640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ab00a3240 0x7f8ab00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.506+0000 7f8abaffd640 1 -- 172.21.15.31:0/2043091286 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f8abc0f10c0 con 0x7f8ac4106f10 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.506+0000 7f8ac9d48640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ab00a3240 0x7f8ab00a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009bd0 tx=0x7f8ac0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.510+0000 7f8abaffd640 1 -- 172.21.15.31:0/2043091286 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8abc0bda60 con 0x7f8ac4106f10 2024-09-16T00:41:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.683+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 --> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ac4075430 con 0x7f8ab00a3240 2024-09-16T00:41:18.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.691+0000 7f8abaffd640 1 -- 172.21.15.31:0/2043091286 <== mgr.14150 v2:172.21.15.31:6800/502885538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340409 (secure 0 0 0) 0x7f8ac4075430 con 0x7f8ab00a3240 2024-09-16T00:41:18.694 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:18.696 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ab00a3240 msgr2=0x7f8ab00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ab00a3240 0x7f8ab00a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009bd0 tx=0x7f8ac0005f90 comp rx=0 tx=0).stop 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 msgr2=0x7f8ac419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 0x7f8ac419eea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8abc031a20 tx=0x7f8abc031a50 comp rx=0 tx=0).stop 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 shutdown_connections 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac4106f10 0x7f8ac419eea0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ac4102790 0x7f8ac419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ab00a3240 0x7f8ab00a5700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 --2- 172.21.15.31:0/2043091286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ac4101de0 0x7f8ac419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 >> 172.21.15.31:0/2043091286 conn(0x7f8ac40fd8c0 msgr2=0x7f8ac40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:18.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 shutdown_connections 2024-09-16T00:41:18.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:18.694+0000 7f8acbfac640 1 -- 172.21.15.31:0/2043091286 wait complete. 2024-09-16T00:41:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:18 smithi081 ceph-mon[35539]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:18.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:18 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:41:19.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:18 smithi102 ceph-mon[35364]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:19.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:18 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:41:19.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:18 smithi031 ceph-mon[30956]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:19.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:18 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:41:19.396 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-16T00:41:17.688771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660841,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":348,"num_read_kb":292,"num_write":277,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":717,"ondisk_log_size":717,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400908,"kb_used_data":14156,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540916,"statfs":{"total":1151940427776,"available":1151529897984,"internally_reserved":0,"allocated":14495744,"data_stored":8185985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5373,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003448"},"pg_stats":[{"pgid":"2.1f","version":"81'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586194+0000","last_change":"2024-09-16T00:38:50.876747+0000","last_active":"2024-09-16T00:40:59.586194+0000","last_peered":"2024-09-16T00:40:59.586194+0000","last_clean":"2024-09-16T00:40:59.586194+0000","last_became_active":"2024-09-16T00:38:50.876388+0000","last_became_peered":"2024-09-16T00:38:50.876388+0000","last_unstale":"2024-09-16T00:40:59.586194+0000","last_undegraded":"2024-09-16T00:40:59.586194+0000","last_fullsized":"2024-09-16T00:40:59.586194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:15.677705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589958+0000","last_change":"2024-09-16T00:40:59.589958+0000","last_active":"2024-09-16T00:40:59.589958+0000","last_peered":"2024-09-16T00:40:59.589958+0000","last_clean":"2024-09-16T00:40:59.589958+0000","last_became_active":"2024-09-16T00:40:59.589549+0000","last_became_peered":"2024-09-16T00:40:59.589549+0000","last_unstale":"2024-09-16T00:40:59.589958+0000","last_undegraded":"2024-09-16T00:40:59.589958+0000","last_fullsized":"2024-09-16T00:40:59.589958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:33.487001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588133+0000","last_change":"2024-09-16T00:39:00.053718+0000","last_active":"2024-09-16T00:40:59.588133+0000","last_peered":"2024-09-16T00:40:59.588133+0000","last_clean":"2024-09-16T00:40:59.588133+0000","last_became_active":"2024-09-16T00:39:00.053319+0000","last_became_peered":"2024-09-16T00:39:00.053319+0000","last_unstale":"2024-09-16T00:40:59.588133+0000","last_undegraded":"2024-09-16T00:40:59.588133+0000","last_fullsized":"2024-09-16T00:40:59.588133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:29.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747727+0000","last_change":"2024-09-16T00:38:50.878322+0000","last_active":"2024-09-16T00:40:59.747727+0000","last_peered":"2024-09-16T00:40:59.747727+0000","last_clean":"2024-09-16T00:40:59.747727+0000","last_became_active":"2024-09-16T00:38:50.877985+0000","last_became_peered":"2024-09-16T00:38:50.877985+0000","last_unstale":"2024-09-16T00:40:59.747727+0000","last_undegraded":"2024-09-16T00:40:59.747727+0000","last_fullsized":"2024-09-16T00:40:59.747727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:00.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162291+0000","last_change":"2024-09-16T00:38:52.974381+0000","last_active":"2024-09-16T00:41:00.162291+0000","last_peered":"2024-09-16T00:41:00.162291+0000","last_clean":"2024-09-16T00:41:00.162291+0000","last_became_active":"2024-09-16T00:38:52.974055+0000","last_became_peered":"2024-09-16T00:38:52.974055+0000","last_unstale":"2024-09-16T00:41:00.162291+0000","last_undegraded":"2024-09-16T00:41:00.162291+0000","last_fullsized":"2024-09-16T00:41:00.162291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:56.780323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589935+0000","last_change":"2024-09-16T00:39:00.050086+0000","last_active":"2024-09-16T00:40:59.589935+0000","last_peered":"2024-09-16T00:40:59.589935+0000","last_clean":"2024-09-16T00:40:59.589935+0000","last_became_active":"2024-09-16T00:39:00.049631+0000","last_became_peered":"2024-09-16T00:39:00.049631+0000","last_unstale":"2024-09-16T00:40:59.589935+0000","last_undegraded":"2024-09-16T00:40:59.589935+0000","last_fullsized":"2024-09-16T00:40:59.589935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:36.228154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586994+0000","last_change":"2024-09-16T00:38:50.877066+0000","last_active":"2024-09-16T00:40:59.586994+0000","last_peered":"2024-09-16T00:40:59.586994+0000","last_clean":"2024-09-16T00:40:59.586994+0000","last_became_active":"2024-09-16T00:38:50.876651+0000","last_became_peered":"2024-09-16T00:38:50.876651+0000","last_unstale":"2024-09-16T00:40:59.586994+0000","last_undegraded":"2024-09-16T00:40:59.586994+0000","last_fullsized":"2024-09-16T00:40:59.586994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:04.268373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587478+0000","last_change":"2024-09-16T00:38:52.964527+0000","last_active":"2024-09-16T00:40:59.587478+0000","last_peered":"2024-09-16T00:40:59.587478+0000","last_clean":"2024-09-16T00:40:59.587478+0000","last_became_active":"2024-09-16T00:38:52.964154+0000","last_became_peered":"2024-09-16T00:38:52.964154+0000","last_unstale":"2024-09-16T00:40:59.587478+0000","last_undegraded":"2024-09-16T00:40:59.587478+0000","last_fullsized":"2024-09-16T00:40:59.587478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:17.849296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590172+0000","last_change":"2024-09-16T00:39:00.048325+0000","last_active":"2024-09-16T00:40:59.590172+0000","last_peered":"2024-09-16T00:40:59.590172+0000","last_clean":"2024-09-16T00:40:59.590172+0000","last_became_active":"2024-09-16T00:39:00.047948+0000","last_became_peered":"2024-09-16T00:39:00.047948+0000","last_unstale":"2024-09-16T00:40:59.590172+0000","last_undegraded":"2024-09-16T00:40:59.590172+0000","last_fullsized":"2024-09-16T00:40:59.590172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:33.118462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587026+0000","last_change":"2024-09-16T00:38:50.871464+0000","last_active":"2024-09-16T00:40:59.587026+0000","last_peered":"2024-09-16T00:40:59.587026+0000","last_clean":"2024-09-16T00:40:59.587026+0000","last_became_active":"2024-09-16T00:38:50.871027+0000","last_became_peered":"2024-09-16T00:38:50.871027+0000","last_unstale":"2024-09-16T00:40:59.587026+0000","last_undegraded":"2024-09-16T00:40:59.587026+0000","last_fullsized":"2024-09-16T00:40:59.587026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:47.203010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.596631+0000","last_change":"2024-09-16T00:40:59.596631+0000","last_active":"2024-09-16T00:40:59.596631+0000","last_peered":"2024-09-16T00:40:59.596631+0000","last_clean":"2024-09-16T00:40:59.596631+0000","last_became_active":"2024-09-16T00:40:59.596165+0000","last_became_peered":"2024-09-16T00:40:59.596165+0000","last_unstale":"2024-09-16T00:40:59.596631+0000","last_undegraded":"2024-09-16T00:40:59.596631+0000","last_fullsized":"2024-09-16T00:40:59.596631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:56.918291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,1],"acting":[9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590072+0000","last_change":"2024-09-16T00:39:00.044978+0000","last_active":"2024-09-16T00:40:59.590072+0000","last_peered":"2024-09-16T00:40:59.590072+0000","last_clean":"2024-09-16T00:40:59.590072+0000","last_became_active":"2024-09-16T00:39:00.044570+0000","last_became_peered":"2024-09-16T00:39:00.044570+0000","last_unstale":"2024-09-16T00:40:59.590072+0000","last_undegraded":"2024-09-16T00:40:59.590072+0000","last_fullsized":"2024-09-16T00:40:59.590072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:02.191965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592530+0000","last_change":"2024-09-16T00:38:52.977036+0000","last_active":"2024-09-16T00:40:59.592530+0000","last_peered":"2024-09-16T00:40:59.592530+0000","last_clean":"2024-09-16T00:40:59.592530+0000","last_became_active":"2024-09-16T00:38:52.976712+0000","last_became_peered":"2024-09-16T00:38:52.976712+0000","last_unstale":"2024-09-16T00:40:59.592530+0000","last_undegraded":"2024-09-16T00:40:59.592530+0000","last_fullsized":"2024-09-16T00:40:59.592530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:53.114733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586270+0000","last_change":"2024-09-16T00:38:50.875959+0000","last_active":"2024-09-16T00:40:59.586270+0000","last_peered":"2024-09-16T00:40:59.586270+0000","last_clean":"2024-09-16T00:40:59.586270+0000","last_became_active":"2024-09-16T00:38:50.875602+0000","last_became_peered":"2024-09-16T00:38:50.875602+0000","last_unstale":"2024-09-16T00:40:59.586270+0000","last_undegraded":"2024-09-16T00:40:59.586270+0000","last_fullsized":"2024-09-16T00:40:59.586270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:39.846815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590295+0000","last_change":"2024-09-16T00:39:00.324852+0000","last_active":"2024-09-16T00:40:59.590295+0000","last_peered":"2024-09-16T00:40:59.590295+0000","last_clean":"2024-09-16T00:40:59.590295+0000","last_became_active":"2024-09-16T00:39:00.324429+0000","last_became_peered":"2024-09-16T00:39:00.324429+0000","last_unstale":"2024-09-16T00:40:59.590295+0000","last_undegraded":"2024-09-16T00:40:59.590295+0000","last_fullsized":"2024-09-16T00:40:59.590295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:12.975083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747373+0000","last_change":"2024-09-16T00:38:52.966901+0000","last_active":"2024-09-16T00:40:59.747373+0000","last_peered":"2024-09-16T00:40:59.747373+0000","last_clean":"2024-09-16T00:40:59.747373+0000","last_became_active":"2024-09-16T00:38:52.966554+0000","last_became_peered":"2024-09-16T00:38:52.966554+0000","last_unstale":"2024-09-16T00:40:59.747373+0000","last_undegraded":"2024-09-16T00:40:59.747373+0000","last_fullsized":"2024-09-16T00:40:59.747373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:57.108282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747467+0000","last_change":"2024-09-16T00:39:59.591063+0000","last_active":"2024-09-16T00:40:59.747467+0000","last_peered":"2024-09-16T00:40:59.747467+0000","last_clean":"2024-09-16T00:40:59.747467+0000","last_became_active":"2024-09-16T00:39:59.590574+0000","last_became_peered":"2024-09-16T00:39:59.590574+0000","last_unstale":"2024-09-16T00:40:59.747467+0000","last_undegraded":"2024-09-16T00:40:59.747467+0000","last_fullsized":"2024-09-16T00:40:59.747467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:18:57.020592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590349+0000","last_change":"2024-09-16T00:39:59.595772+0000","last_active":"2024-09-16T00:40:59.590349+0000","last_peered":"2024-09-16T00:40:59.590349+0000","last_clean":"2024-09-16T00:40:59.590349+0000","last_became_active":"2024-09-16T00:39:59.590772+0000","last_became_peered":"2024-09-16T00:39:59.590772+0000","last_unstale":"2024-09-16T00:40:59.590349+0000","last_undegraded":"2024-09-16T00:40:59.590349+0000","last_fullsized":"2024-09-16T00:40:59.590349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:53.236019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747573+0000","last_change":"2024-09-16T00:38:52.964913+0000","last_active":"2024-09-16T00:40:59.747573+0000","last_peered":"2024-09-16T00:40:59.747573+0000","last_clean":"2024-09-16T00:40:59.747573+0000","last_became_active":"2024-09-16T00:38:52.964521+0000","last_became_peered":"2024-09-16T00:38:52.964521+0000","last_unstale":"2024-09-16T00:40:59.747573+0000","last_undegraded":"2024-09-16T00:40:59.747573+0000","last_fullsized":"2024-09-16T00:40:59.747573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:32.782092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747545+0000","last_change":"2024-09-16T00:38:50.871573+0000","last_active":"2024-09-16T00:40:59.747545+0000","last_peered":"2024-09-16T00:40:59.747545+0000","last_clean":"2024-09-16T00:40:59.747545+0000","last_became_active":"2024-09-16T00:38:50.871191+0000","last_became_peered":"2024-09-16T00:38:50.871191+0000","last_unstale":"2024-09-16T00:40:59.747545+0000","last_undegraded":"2024-09-16T00:40:59.747545+0000","last_fullsized":"2024-09-16T00:40:59.747545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:18.821992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585893+0000","last_change":"2024-09-16T00:39:59.386827+0000","last_active":"2024-09-16T00:40:59.585893+0000","last_peered":"2024-09-16T00:40:59.585893+0000","last_clean":"2024-09-16T00:40:59.585893+0000","last_became_active":"2024-09-16T00:39:59.386389+0000","last_became_peered":"2024-09-16T00:39:59.386389+0000","last_unstale":"2024-09-16T00:40:59.585893+0000","last_undegraded":"2024-09-16T00:40:59.585893+0000","last_fullsized":"2024-09-16T00:40:59.585893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:12:21.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585940+0000","last_change":"2024-09-16T00:38:52.973011+0000","last_active":"2024-09-16T00:40:59.585940+0000","last_peered":"2024-09-16T00:40:59.585940+0000","last_clean":"2024-09-16T00:40:59.585940+0000","last_became_active":"2024-09-16T00:38:52.971869+0000","last_became_peered":"2024-09-16T00:38:52.971869+0000","last_unstale":"2024-09-16T00:40:59.585940+0000","last_undegraded":"2024-09-16T00:40:59.585940+0000","last_fullsized":"2024-09-16T00:40:59.585940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.885506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587363+0000","last_change":"2024-09-16T00:38:50.878375+0000","last_active":"2024-09-16T00:40:59.587363+0000","last_peered":"2024-09-16T00:40:59.587363+0000","last_clean":"2024-09-16T00:40:59.587363+0000","last_became_active":"2024-09-16T00:38:50.878050+0000","last_became_peered":"2024-09-16T00:38:50.878050+0000","last_unstale":"2024-09-16T00:40:59.587363+0000","last_undegraded":"2024-09-16T00:40:59.587363+0000","last_fullsized":"2024-09-16T00:40:59.587363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.348457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590242+0000","last_change":"2024-09-16T00:39:00.326089+0000","last_active":"2024-09-16T00:40:59.590242+0000","last_peered":"2024-09-16T00:40:59.590242+0000","last_clean":"2024-09-16T00:40:59.590242+0000","last_became_active":"2024-09-16T00:39:00.325282+0000","last_became_peered":"2024-09-16T00:39:00.325282+0000","last_unstale":"2024-09-16T00:40:59.590242+0000","last_undegraded":"2024-09-16T00:40:59.590242+0000","last_fullsized":"2024-09-16T00:40:59.590242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:48.242569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162114+0000","last_change":"2024-09-16T00:38:52.964957+0000","last_active":"2024-09-16T00:41:00.162114+0000","last_peered":"2024-09-16T00:41:00.162114+0000","last_clean":"2024-09-16T00:41:00.162114+0000","last_became_active":"2024-09-16T00:38:52.964640+0000","last_became_peered":"2024-09-16T00:38:52.964640+0000","last_unstale":"2024-09-16T00:41:00.162114+0000","last_undegraded":"2024-09-16T00:41:00.162114+0000","last_fullsized":"2024-09-16T00:41:00.162114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.499440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590740+0000","last_change":"2024-09-16T00:38:51.423477+0000","last_active":"2024-09-16T00:40:59.590740+0000","last_peered":"2024-09-16T00:40:59.590740+0000","last_clean":"2024-09-16T00:40:59.590740+0000","last_became_active":"2024-09-16T00:38:51.423119+0000","last_became_peered":"2024-09-16T00:38:51.423119+0000","last_unstale":"2024-09-16T00:40:59.590740+0000","last_undegraded":"2024-09-16T00:40:59.590740+0000","last_fullsized":"2024-09-16T00:40:59.590740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:25.302403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588415+0000","last_change":"2024-09-16T00:39:00.052543+0000","last_active":"2024-09-16T00:40:59.588415+0000","last_peered":"2024-09-16T00:40:59.588415+0000","last_clean":"2024-09-16T00:40:59.588415+0000","last_became_active":"2024-09-16T00:39:00.052108+0000","last_became_peered":"2024-09-16T00:39:00.052108+0000","last_unstale":"2024-09-16T00:40:59.588415+0000","last_undegraded":"2024-09-16T00:40:59.588415+0000","last_fullsized":"2024-09-16T00:40:59.588415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:12.812443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747653+0000","last_change":"2024-09-16T00:38:52.970163+0000","last_active":"2024-09-16T00:40:59.747653+0000","last_peered":"2024-09-16T00:40:59.747653+0000","last_clean":"2024-09-16T00:40:59.747653+0000","last_became_active":"2024-09-16T00:38:52.969683+0000","last_became_peered":"2024-09-16T00:38:52.969683+0000","last_unstale":"2024-09-16T00:40:59.747653+0000","last_undegraded":"2024-09-16T00:40:59.747653+0000","last_fullsized":"2024-09-16T00:40:59.747653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:29.823124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:04.350207+0000","last_change":"2024-09-16T00:38:50.869613+0000","last_active":"2024-09-16T00:41:04.350207+0000","last_peered":"2024-09-16T00:41:04.350207+0000","last_clean":"2024-09-16T00:41:04.350207+0000","last_became_active":"2024-09-16T00:38:50.869297+0000","last_became_peered":"2024-09-16T00:38:50.869297+0000","last_unstale":"2024-09-16T00:41:04.350207+0000","last_undegraded":"2024-09-16T00:41:04.350207+0000","last_fullsized":"2024-09-16T00:41:04.350207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:00.868229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747093+0000","last_change":"2024-09-16T00:39:00.324761+0000","last_active":"2024-09-16T00:40:59.747093+0000","last_peered":"2024-09-16T00:40:59.747093+0000","last_clean":"2024-09-16T00:40:59.747093+0000","last_became_active":"2024-09-16T00:39:00.324304+0000","last_became_peered":"2024-09-16T00:39:00.324304+0000","last_unstale":"2024-09-16T00:40:59.747093+0000","last_undegraded":"2024-09-16T00:40:59.747093+0000","last_fullsized":"2024-09-16T00:40:59.747093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:58.680694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595527+0000","last_change":"2024-09-16T00:38:52.964200+0000","last_active":"2024-09-16T00:40:59.595527+0000","last_peered":"2024-09-16T00:40:59.595527+0000","last_clean":"2024-09-16T00:40:59.595527+0000","last_became_active":"2024-09-16T00:38:52.963896+0000","last_became_peered":"2024-09-16T00:38:52.963896+0000","last_unstale":"2024-09-16T00:40:59.595527+0000","last_undegraded":"2024-09-16T00:40:59.595527+0000","last_fullsized":"2024-09-16T00:40:59.595527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:52.081495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587025+0000","last_change":"2024-09-16T00:38:50.878952+0000","last_active":"2024-09-16T00:40:59.587025+0000","last_peered":"2024-09-16T00:40:59.587025+0000","last_clean":"2024-09-16T00:40:59.587025+0000","last_became_active":"2024-09-16T00:38:50.878447+0000","last_became_peered":"2024-09-16T00:38:50.878447+0000","last_unstale":"2024-09-16T00:40:59.587025+0000","last_undegraded":"2024-09-16T00:40:59.587025+0000","last_fullsized":"2024-09-16T00:40:59.587025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:10.790250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747410+0000","last_change":"2024-09-16T00:39:00.048411+0000","last_active":"2024-09-16T00:40:59.747410+0000","last_peered":"2024-09-16T00:40:59.747410+0000","last_clean":"2024-09-16T00:40:59.747410+0000","last_became_active":"2024-09-16T00:39:00.048001+0000","last_became_peered":"2024-09-16T00:39:00.048001+0000","last_unstale":"2024-09-16T00:40:59.747410+0000","last_undegraded":"2024-09-16T00:40:59.747410+0000","last_fullsized":"2024-09-16T00:40:59.747410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:16.970373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747586+0000","last_change":"2024-09-16T00:38:52.956389+0000","last_active":"2024-09-16T00:40:59.747586+0000","last_peered":"2024-09-16T00:40:59.747586+0000","last_clean":"2024-09-16T00:40:59.747586+0000","last_became_active":"2024-09-16T00:38:52.955968+0000","last_became_peered":"2024-09-16T00:38:52.955968+0000","last_unstale":"2024-09-16T00:40:59.747586+0000","last_undegraded":"2024-09-16T00:40:59.747586+0000","last_fullsized":"2024-09-16T00:40:59.747586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:02.800423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585296+0000","last_change":"2024-09-16T00:38:50.878891+0000","last_active":"2024-09-16T00:40:59.585296+0000","last_peered":"2024-09-16T00:40:59.585296+0000","last_clean":"2024-09-16T00:40:59.585296+0000","last_became_active":"2024-09-16T00:38:50.878544+0000","last_became_peered":"2024-09-16T00:38:50.878544+0000","last_unstale":"2024-09-16T00:40:59.585296+0000","last_undegraded":"2024-09-16T00:40:59.585296+0000","last_fullsized":"2024-09-16T00:40:59.585296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:57.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586050+0000","last_change":"2024-09-16T00:39:00.325258+0000","last_active":"2024-09-16T00:40:59.586050+0000","last_peered":"2024-09-16T00:40:59.586050+0000","last_clean":"2024-09-16T00:40:59.586050+0000","last_became_active":"2024-09-16T00:39:00.324716+0000","last_became_peered":"2024-09-16T00:39:00.324716+0000","last_unstale":"2024-09-16T00:40:59.586050+0000","last_undegraded":"2024-09-16T00:40:59.586050+0000","last_fullsized":"2024-09-16T00:40:59.586050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:26.988965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747487+0000","last_change":"2024-09-16T00:38:52.966710+0000","last_active":"2024-09-16T00:40:59.747487+0000","last_peered":"2024-09-16T00:40:59.747487+0000","last_clean":"2024-09-16T00:40:59.747487+0000","last_became_active":"2024-09-16T00:38:52.966385+0000","last_became_peered":"2024-09-16T00:38:52.966385+0000","last_unstale":"2024-09-16T00:40:59.747487+0000","last_undegraded":"2024-09-16T00:40:59.747487+0000","last_fullsized":"2024-09-16T00:40:59.747487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:09.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592602+0000","last_change":"2024-09-16T00:38:50.873307+0000","last_active":"2024-09-16T00:40:59.592602+0000","last_peered":"2024-09-16T00:40:59.592602+0000","last_clean":"2024-09-16T00:40:59.592602+0000","last_became_active":"2024-09-16T00:38:50.872973+0000","last_became_peered":"2024-09-16T00:38:50.872973+0000","last_unstale":"2024-09-16T00:40:59.592602+0000","last_undegraded":"2024-09-16T00:40:59.592602+0000","last_fullsized":"2024-09-16T00:40:59.592602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:27.721295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595268+0000","last_change":"2024-09-16T00:39:00.050308+0000","last_active":"2024-09-16T00:40:59.595268+0000","last_peered":"2024-09-16T00:40:59.595268+0000","last_clean":"2024-09-16T00:40:59.595268+0000","last_became_active":"2024-09-16T00:39:00.049275+0000","last_became_peered":"2024-09-16T00:39:00.049275+0000","last_unstale":"2024-09-16T00:40:59.595268+0000","last_undegraded":"2024-09-16T00:40:59.595268+0000","last_fullsized":"2024-09-16T00:40:59.595268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:01:27.790613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.591482+0000","last_change":"2024-09-16T00:40:59.591482+0000","last_active":"2024-09-16T00:40:59.591482+0000","last_peered":"2024-09-16T00:40:59.591482+0000","last_clean":"2024-09-16T00:40:59.591482+0000","last_became_active":"2024-09-16T00:40:59.590978+0000","last_became_peered":"2024-09-16T00:40:59.590978+0000","last_unstale":"2024-09-16T00:40:59.591482+0000","last_undegraded":"2024-09-16T00:40:59.591482+0000","last_fullsized":"2024-09-16T00:40:59.591482+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:56.975117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594993+0000","last_change":"2024-09-16T00:38:50.878285+0000","last_active":"2024-09-16T00:40:59.594993+0000","last_peered":"2024-09-16T00:40:59.594993+0000","last_clean":"2024-09-16T00:40:59.594993+0000","last_became_active":"2024-09-16T00:38:50.877931+0000","last_became_peered":"2024-09-16T00:38:50.877931+0000","last_unstale":"2024-09-16T00:40:59.594993+0000","last_undegraded":"2024-09-16T00:40:59.594993+0000","last_fullsized":"2024-09-16T00:40:59.594993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:31.522678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595391+0000","last_change":"2024-09-16T00:39:00.324766+0000","last_active":"2024-09-16T00:40:59.595391+0000","last_peered":"2024-09-16T00:40:59.595391+0000","last_clean":"2024-09-16T00:40:59.595391+0000","last_became_active":"2024-09-16T00:39:00.324287+0000","last_became_peered":"2024-09-16T00:39:00.324287+0000","last_unstale":"2024-09-16T00:40:59.595391+0000","last_undegraded":"2024-09-16T00:40:59.595391+0000","last_fullsized":"2024-09-16T00:40:59.595391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:40.173279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594855+0000","last_change":"2024-09-16T00:38:52.966760+0000","last_active":"2024-09-16T00:40:59.594855+0000","last_peered":"2024-09-16T00:40:59.594855+0000","last_clean":"2024-09-16T00:40:59.594855+0000","last_became_active":"2024-09-16T00:38:52.966427+0000","last_became_peered":"2024-09-16T00:38:52.966427+0000","last_unstale":"2024-09-16T00:40:59.594855+0000","last_undegraded":"2024-09-16T00:40:59.594855+0000","last_fullsized":"2024-09-16T00:40:59.594855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:23.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747209+0000","last_change":"2024-09-16T00:39:59.376963+0000","last_active":"2024-09-16T00:40:59.747209+0000","last_peered":"2024-09-16T00:40:59.747209+0000","last_clean":"2024-09-16T00:40:59.747209+0000","last_became_active":"2024-09-16T00:39:59.376538+0000","last_became_peered":"2024-09-16T00:39:59.376538+0000","last_unstale":"2024-09-16T00:40:59.747209+0000","last_undegraded":"2024-09-16T00:40:59.747209+0000","last_fullsized":"2024-09-16T00:40:59.747209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:02.077749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587907+0000","last_change":"2024-09-16T00:39:00.326305+0000","last_active":"2024-09-16T00:40:59.587907+0000","last_peered":"2024-09-16T00:40:59.587907+0000","last_clean":"2024-09-16T00:40:59.587907+0000","last_became_active":"2024-09-16T00:39:00.325909+0000","last_became_peered":"2024-09-16T00:39:00.325909+0000","last_unstale":"2024-09-16T00:40:59.587907+0000","last_undegraded":"2024-09-16T00:40:59.587907+0000","last_fullsized":"2024-09-16T00:40:59.587907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:29.026548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747729+0000","last_change":"2024-09-16T00:38:52.968703+0000","last_active":"2024-09-16T00:40:59.747729+0000","last_peered":"2024-09-16T00:40:59.747729+0000","last_clean":"2024-09-16T00:40:59.747729+0000","last_became_active":"2024-09-16T00:38:52.968388+0000","last_became_peered":"2024-09-16T00:38:52.968388+0000","last_unstale":"2024-09-16T00:40:59.747729+0000","last_undegraded":"2024-09-16T00:40:59.747729+0000","last_fullsized":"2024-09-16T00:40:59.747729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:03.674068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590591+0000","last_change":"2024-09-16T00:38:50.872416+0000","last_active":"2024-09-16T00:40:59.590591+0000","last_peered":"2024-09-16T00:40:59.590591+0000","last_clean":"2024-09-16T00:40:59.590591+0000","last_became_active":"2024-09-16T00:38:50.872029+0000","last_became_peered":"2024-09-16T00:38:50.872029+0000","last_unstale":"2024-09-16T00:40:59.590591+0000","last_undegraded":"2024-09-16T00:40:59.590591+0000","last_fullsized":"2024-09-16T00:40:59.590591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:00.702195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592357+0000","last_change":"2024-09-16T00:37:37.427899+0000","last_active":"2024-09-16T00:40:59.592357+0000","last_peered":"2024-09-16T00:40:59.592357+0000","last_clean":"2024-09-16T00:40:59.592357+0000","last_became_active":"2024-09-16T00:37:37.427445+0000","last_became_peered":"2024-09-16T00:37:37.427445+0000","last_unstale":"2024-09-16T00:40:59.592357+0000","last_undegraded":"2024-09-16T00:40:59.592357+0000","last_fullsized":"2024-09-16T00:40:59.592357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:34:12.449127+0000","last_clean_scrub_stamp":"2024-09-16T00:34:12.449127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:13.228455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592743+0000","last_change":"2024-09-16T00:39:00.049450+0000","last_active":"2024-09-16T00:40:59.592743+0000","last_peered":"2024-09-16T00:40:59.592743+0000","last_clean":"2024-09-16T00:40:59.592743+0000","last_became_active":"2024-09-16T00:39:00.049033+0000","last_became_peered":"2024-09-16T00:39:00.049033+0000","last_unstale":"2024-09-16T00:40:59.592743+0000","last_undegraded":"2024-09-16T00:40:59.592743+0000","last_fullsized":"2024-09-16T00:40:59.592743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:01.367141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592923+0000","last_change":"2024-09-16T00:38:52.975676+0000","last_active":"2024-09-16T00:40:59.592923+0000","last_peered":"2024-09-16T00:40:59.592923+0000","last_clean":"2024-09-16T00:40:59.592923+0000","last_became_active":"2024-09-16T00:38:52.975309+0000","last_became_peered":"2024-09-16T00:38:52.975309+0000","last_unstale":"2024-09-16T00:40:59.592923+0000","last_undegraded":"2024-09-16T00:40:59.592923+0000","last_fullsized":"2024-09-16T00:40:59.592923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:17.589161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589988+0000","last_change":"2024-09-16T00:39:59.593442+0000","last_active":"2024-09-16T00:40:59.589988+0000","last_peered":"2024-09-16T00:40:59.589988+0000","last_clean":"2024-09-16T00:40:59.589988+0000","last_became_active":"2024-09-16T00:39:59.593039+0000","last_became_peered":"2024-09-16T00:39:59.593039+0000","last_unstale":"2024-09-16T00:40:59.589988+0000","last_undegraded":"2024-09-16T00:40:59.589988+0000","last_fullsized":"2024-09-16T00:40:59.589988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:36.644148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587371+0000","last_change":"2024-09-16T00:39:00.041362+0000","last_active":"2024-09-16T00:40:59.587371+0000","last_peered":"2024-09-16T00:40:59.587371+0000","last_clean":"2024-09-16T00:40:59.587371+0000","last_became_active":"2024-09-16T00:39:00.040853+0000","last_became_peered":"2024-09-16T00:39:00.040853+0000","last_unstale":"2024-09-16T00:40:59.587371+0000","last_undegraded":"2024-09-16T00:40:59.587371+0000","last_fullsized":"2024-09-16T00:40:59.587371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:20.125630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592697+0000","last_change":"2024-09-16T00:38:52.972264+0000","last_active":"2024-09-16T00:40:59.592697+0000","last_peered":"2024-09-16T00:40:59.592697+0000","last_clean":"2024-09-16T00:40:59.592697+0000","last_became_active":"2024-09-16T00:38:52.971954+0000","last_became_peered":"2024-09-16T00:38:52.971954+0000","last_unstale":"2024-09-16T00:40:59.592697+0000","last_undegraded":"2024-09-16T00:40:59.592697+0000","last_fullsized":"2024-09-16T00:40:59.592697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:40.188122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589435+0000","last_change":"2024-09-16T00:38:50.872272+0000","last_active":"2024-09-16T00:40:59.589435+0000","last_peered":"2024-09-16T00:40:59.589435+0000","last_clean":"2024-09-16T00:40:59.589435+0000","last_became_active":"2024-09-16T00:38:50.871881+0000","last_became_peered":"2024-09-16T00:38:50.871881+0000","last_unstale":"2024-09-16T00:40:59.589435+0000","last_undegraded":"2024-09-16T00:40:59.589435+0000","last_fullsized":"2024-09-16T00:40:59.589435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:25.617705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161336+0000","last_change":"2024-09-16T00:39:00.048170+0000","last_active":"2024-09-16T00:41:00.161336+0000","last_peered":"2024-09-16T00:41:00.161336+0000","last_clean":"2024-09-16T00:41:00.161336+0000","last_became_active":"2024-09-16T00:39:00.047789+0000","last_became_peered":"2024-09-16T00:39:00.047789+0000","last_unstale":"2024-09-16T00:41:00.161336+0000","last_undegraded":"2024-09-16T00:41:00.161336+0000","last_fullsized":"2024-09-16T00:41:00.161336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:17.057316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592993+0000","last_change":"2024-09-16T00:38:52.974516+0000","last_active":"2024-09-16T00:40:59.592993+0000","last_peered":"2024-09-16T00:40:59.592993+0000","last_clean":"2024-09-16T00:40:59.592993+0000","last_became_active":"2024-09-16T00:38:52.974180+0000","last_became_peered":"2024-09-16T00:38:52.974180+0000","last_unstale":"2024-09-16T00:40:59.592993+0000","last_undegraded":"2024-09-16T00:40:59.592993+0000","last_fullsized":"2024-09-16T00:40:59.592993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:34.558505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595517+0000","last_change":"2024-09-16T00:39:59.594994+0000","last_active":"2024-09-16T00:40:59.595517+0000","last_peered":"2024-09-16T00:40:59.595517+0000","last_clean":"2024-09-16T00:40:59.595517+0000","last_became_active":"2024-09-16T00:39:59.590640+0000","last_became_peered":"2024-09-16T00:39:59.590640+0000","last_unstale":"2024-09-16T00:40:59.595517+0000","last_undegraded":"2024-09-16T00:40:59.595517+0000","last_fullsized":"2024-09-16T00:40:59.595517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:21.391494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587245+0000","last_change":"2024-09-16T00:39:00.053949+0000","last_active":"2024-09-16T00:40:59.587245+0000","last_peered":"2024-09-16T00:40:59.587245+0000","last_clean":"2024-09-16T00:40:59.587245+0000","last_became_active":"2024-09-16T00:39:00.053163+0000","last_became_peered":"2024-09-16T00:39:00.053163+0000","last_unstale":"2024-09-16T00:40:59.587245+0000","last_undegraded":"2024-09-16T00:40:59.587245+0000","last_fullsized":"2024-09-16T00:40:59.587245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:01.525483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747566+0000","last_change":"2024-09-16T00:38:52.968960+0000","last_active":"2024-09-16T00:40:59.747566+0000","last_peered":"2024-09-16T00:40:59.747566+0000","last_clean":"2024-09-16T00:40:59.747566+0000","last_became_active":"2024-09-16T00:38:52.968535+0000","last_became_peered":"2024-09-16T00:38:52.968535+0000","last_unstale":"2024-09-16T00:40:59.747566+0000","last_undegraded":"2024-09-16T00:40:59.747566+0000","last_fullsized":"2024-09-16T00:40:59.747566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:57.985865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586785+0000","last_change":"2024-09-16T00:38:50.871723+0000","last_active":"2024-09-16T00:40:59.586785+0000","last_peered":"2024-09-16T00:40:59.586785+0000","last_clean":"2024-09-16T00:40:59.586785+0000","last_became_active":"2024-09-16T00:38:50.871391+0000","last_became_peered":"2024-09-16T00:38:50.871391+0000","last_unstale":"2024-09-16T00:40:59.586785+0000","last_undegraded":"2024-09-16T00:40:59.586785+0000","last_fullsized":"2024-09-16T00:40:59.586785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:21.475081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588117+0000","last_change":"2024-09-16T00:39:00.049241+0000","last_active":"2024-09-16T00:40:59.588117+0000","last_peered":"2024-09-16T00:40:59.588117+0000","last_clean":"2024-09-16T00:40:59.588117+0000","last_became_active":"2024-09-16T00:39:00.048171+0000","last_became_peered":"2024-09-16T00:39:00.048171+0000","last_unstale":"2024-09-16T00:40:59.588117+0000","last_undegraded":"2024-09-16T00:40:59.588117+0000","last_fullsized":"2024-09-16T00:40:59.588117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:32.927890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588295+0000","last_change":"2024-09-16T00:38:52.971986+0000","last_active":"2024-09-16T00:40:59.588295+0000","last_peered":"2024-09-16T00:40:59.588295+0000","last_clean":"2024-09-16T00:40:59.588295+0000","last_became_active":"2024-09-16T00:38:52.971146+0000","last_became_peered":"2024-09-16T00:38:52.971146+0000","last_unstale":"2024-09-16T00:40:59.588295+0000","last_undegraded":"2024-09-16T00:40:59.588295+0000","last_fullsized":"2024-09-16T00:40:59.588295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.983293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585921+0000","last_change":"2024-09-16T00:38:50.879389+0000","last_active":"2024-09-16T00:40:59.585921+0000","last_peered":"2024-09-16T00:40:59.585921+0000","last_clean":"2024-09-16T00:40:59.585921+0000","last_became_active":"2024-09-16T00:38:50.878324+0000","last_became_peered":"2024-09-16T00:38:50.878324+0000","last_unstale":"2024-09-16T00:40:59.585921+0000","last_undegraded":"2024-09-16T00:40:59.585921+0000","last_fullsized":"2024-09-16T00:40:59.585921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:02.317202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595364+0000","last_change":"2024-09-16T00:39:00.047609+0000","last_active":"2024-09-16T00:40:59.595364+0000","last_peered":"2024-09-16T00:40:59.595364+0000","last_clean":"2024-09-16T00:40:59.595364+0000","last_became_active":"2024-09-16T00:39:00.047130+0000","last_became_peered":"2024-09-16T00:39:00.047130+0000","last_unstale":"2024-09-16T00:40:59.595364+0000","last_undegraded":"2024-09-16T00:40:59.595364+0000","last_fullsized":"2024-09-16T00:40:59.595364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:38.747663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592245+0000","last_change":"2024-09-16T00:38:52.976584+0000","last_active":"2024-09-16T00:40:59.592245+0000","last_peered":"2024-09-16T00:40:59.592245+0000","last_clean":"2024-09-16T00:40:59.592245+0000","last_became_active":"2024-09-16T00:38:52.976248+0000","last_became_peered":"2024-09-16T00:38:52.976248+0000","last_unstale":"2024-09-16T00:40:59.592245+0000","last_undegraded":"2024-09-16T00:40:59.592245+0000","last_fullsized":"2024-09-16T00:40:59.592245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:00.792698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590466+0000","last_change":"2024-09-16T00:38:50.876092+0000","last_active":"2024-09-16T00:40:59.590466+0000","last_peered":"2024-09-16T00:40:59.590466+0000","last_clean":"2024-09-16T00:40:59.590466+0000","last_became_active":"2024-09-16T00:38:50.875648+0000","last_became_peered":"2024-09-16T00:38:50.875648+0000","last_unstale":"2024-09-16T00:40:59.590466+0000","last_undegraded":"2024-09-16T00:40:59.590466+0000","last_fullsized":"2024-09-16T00:40:59.590466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:18.485521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586021+0000","last_change":"2024-09-16T00:39:59.593568+0000","last_active":"2024-09-16T00:40:59.586021+0000","last_peered":"2024-09-16T00:40:59.586021+0000","last_clean":"2024-09-16T00:40:59.586021+0000","last_became_active":"2024-09-16T00:39:59.593209+0000","last_became_peered":"2024-09-16T00:39:59.593209+0000","last_unstale":"2024-09-16T00:40:59.586021+0000","last_undegraded":"2024-09-16T00:40:59.586021+0000","last_fullsized":"2024-09-16T00:40:59.586021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:56.516773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.597069+0000","last_change":"2024-09-16T00:40:59.596544+0000","last_active":"2024-09-16T00:40:59.597069+0000","last_peered":"2024-09-16T00:40:59.597069+0000","last_clean":"2024-09-16T00:40:59.597069+0000","last_became_active":"2024-09-16T00:40:59.596128+0000","last_became_peered":"2024-09-16T00:40:59.596128+0000","last_unstale":"2024-09-16T00:40:59.597069+0000","last_undegraded":"2024-09-16T00:40:59.597069+0000","last_fullsized":"2024-09-16T00:40:59.597069+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:22.691164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587694+0000","last_change":"2024-09-16T00:38:50.873524+0000","last_active":"2024-09-16T00:40:59.587694+0000","last_peered":"2024-09-16T00:40:59.587694+0000","last_clean":"2024-09-16T00:40:59.587694+0000","last_became_active":"2024-09-16T00:38:50.873085+0000","last_became_peered":"2024-09-16T00:38:50.873085+0000","last_unstale":"2024-09-16T00:40:59.587694+0000","last_undegraded":"2024-09-16T00:40:59.587694+0000","last_fullsized":"2024-09-16T00:40:59.587694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:36.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747362+0000","last_change":"2024-09-16T00:39:59.380258+0000","last_active":"2024-09-16T00:40:59.747362+0000","last_peered":"2024-09-16T00:40:59.747362+0000","last_clean":"2024-09-16T00:40:59.747362+0000","last_became_active":"2024-09-16T00:39:59.379813+0000","last_became_peered":"2024-09-16T00:39:59.379813+0000","last_unstale":"2024-09-16T00:40:59.747362+0000","last_undegraded":"2024-09-16T00:40:59.747362+0000","last_fullsized":"2024-09-16T00:40:59.747362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:03.110305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595116+0000","last_change":"2024-09-16T00:38:52.965611+0000","last_active":"2024-09-16T00:40:59.595116+0000","last_peered":"2024-09-16T00:40:59.595116+0000","last_clean":"2024-09-16T00:40:59.595116+0000","last_became_active":"2024-09-16T00:38:52.965322+0000","last_became_peered":"2024-09-16T00:38:52.965322+0000","last_unstale":"2024-09-16T00:40:59.595116+0000","last_undegraded":"2024-09-16T00:40:59.595116+0000","last_fullsized":"2024-09-16T00:40:59.595116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:26.419307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588686+0000","last_change":"2024-09-16T00:38:50.873662+0000","last_active":"2024-09-16T00:40:59.588686+0000","last_peered":"2024-09-16T00:40:59.588686+0000","last_clean":"2024-09-16T00:40:59.588686+0000","last_became_active":"2024-09-16T00:38:50.873188+0000","last_became_peered":"2024-09-16T00:38:50.873188+0000","last_unstale":"2024-09-16T00:40:59.588686+0000","last_undegraded":"2024-09-16T00:40:59.588686+0000","last_fullsized":"2024-09-16T00:40:59.588686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:04.775986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588982+0000","last_change":"2024-09-16T00:39:00.049576+0000","last_active":"2024-09-16T00:40:59.588982+0000","last_peered":"2024-09-16T00:40:59.588982+0000","last_clean":"2024-09-16T00:40:59.588982+0000","last_became_active":"2024-09-16T00:39:00.049157+0000","last_became_peered":"2024-09-16T00:39:00.049157+0000","last_unstale":"2024-09-16T00:40:59.588982+0000","last_undegraded":"2024-09-16T00:40:59.588982+0000","last_fullsized":"2024-09-16T00:40:59.588982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:13.751576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586881+0000","last_change":"2024-09-16T00:38:52.970383+0000","last_active":"2024-09-16T00:40:59.586881+0000","last_peered":"2024-09-16T00:40:59.586881+0000","last_clean":"2024-09-16T00:40:59.586881+0000","last_became_active":"2024-09-16T00:38:52.970058+0000","last_became_peered":"2024-09-16T00:38:52.970058+0000","last_unstale":"2024-09-16T00:40:59.586881+0000","last_undegraded":"2024-09-16T00:40:59.586881+0000","last_fullsized":"2024-09-16T00:40:59.586881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:28.824490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":289,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:15.306590+0000","last_change":"2024-09-16T00:38:50.873581+0000","last_active":"2024-09-16T00:41:15.306590+0000","last_peered":"2024-09-16T00:41:15.306590+0000","last_clean":"2024-09-16T00:41:15.306590+0000","last_became_active":"2024-09-16T00:38:50.873183+0000","last_became_peered":"2024-09-16T00:38:50.873183+0000","last_unstale":"2024-09-16T00:41:15.306590+0000","last_undegraded":"2024-09-16T00:41:15.306590+0000","last_fullsized":"2024-09-16T00:41:15.306590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:12.940409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69441,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":215,"num_read_kb":176,"num_write":35,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.585067+0000","last_change":"2024-09-16T00:39:00.054411+0000","last_active":"2024-09-16T00:40:59.585067+0000","last_peered":"2024-09-16T00:40:59.585067+0000","last_clean":"2024-09-16T00:40:59.585067+0000","last_became_active":"2024-09-16T00:39:00.053970+0000","last_became_peered":"2024-09-16T00:39:00.053970+0000","last_unstale":"2024-09-16T00:40:59.585067+0000","last_undegraded":"2024-09-16T00:40:59.585067+0000","last_fullsized":"2024-09-16T00:40:59.585067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:14.130938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587633+0000","last_change":"2024-09-16T00:38:52.972435+0000","last_active":"2024-09-16T00:40:59.587633+0000","last_peered":"2024-09-16T00:40:59.587633+0000","last_clean":"2024-09-16T00:40:59.587633+0000","last_became_active":"2024-09-16T00:38:52.972128+0000","last_became_peered":"2024-09-16T00:38:52.972128+0000","last_unstale":"2024-09-16T00:40:59.587633+0000","last_undegraded":"2024-09-16T00:40:59.587633+0000","last_fullsized":"2024-09-16T00:40:59.587633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:29.091949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.589321+0000","last_change":"2024-09-16T00:38:50.878903+0000","last_active":"2024-09-16T00:40:59.589321+0000","last_peered":"2024-09-16T00:40:59.589321+0000","last_clean":"2024-09-16T00:40:59.589321+0000","last_became_active":"2024-09-16T00:38:50.878522+0000","last_became_peered":"2024-09-16T00:38:50.878522+0000","last_unstale":"2024-09-16T00:40:59.589321+0000","last_undegraded":"2024-09-16T00:40:59.589321+0000","last_fullsized":"2024-09-16T00:40:59.589321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:09.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590297+0000","last_change":"2024-09-16T00:39:00.050619+0000","last_active":"2024-09-16T00:40:59.590297+0000","last_peered":"2024-09-16T00:40:59.590297+0000","last_clean":"2024-09-16T00:40:59.590297+0000","last_became_active":"2024-09-16T00:39:00.049292+0000","last_became_peered":"2024-09-16T00:39:00.049292+0000","last_unstale":"2024-09-16T00:40:59.590297+0000","last_undegraded":"2024-09-16T00:40:59.590297+0000","last_fullsized":"2024-09-16T00:40:59.590297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:05.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162296+0000","last_change":"2024-09-16T00:38:52.970314+0000","last_active":"2024-09-16T00:41:00.162296+0000","last_peered":"2024-09-16T00:41:00.162296+0000","last_clean":"2024-09-16T00:41:00.162296+0000","last_became_active":"2024-09-16T00:38:52.970011+0000","last_became_peered":"2024-09-16T00:38:52.970011+0000","last_unstale":"2024-09-16T00:41:00.162296+0000","last_undegraded":"2024-09-16T00:41:00.162296+0000","last_fullsized":"2024-09-16T00:41:00.162296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:23:06.034116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592831+0000","last_change":"2024-09-16T00:38:50.876798+0000","last_active":"2024-09-16T00:40:59.592831+0000","last_peered":"2024-09-16T00:40:59.592831+0000","last_clean":"2024-09-16T00:40:59.592831+0000","last_became_active":"2024-09-16T00:38:50.876436+0000","last_became_peered":"2024-09-16T00:38:50.876436+0000","last_unstale":"2024-09-16T00:40:59.592831+0000","last_undegraded":"2024-09-16T00:40:59.592831+0000","last_fullsized":"2024-09-16T00:40:59.592831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:14.202199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584869+0000","last_change":"2024-09-16T00:39:00.052003+0000","last_active":"2024-09-16T00:40:59.584869+0000","last_peered":"2024-09-16T00:40:59.584869+0000","last_clean":"2024-09-16T00:40:59.584869+0000","last_became_active":"2024-09-16T00:39:00.051608+0000","last_became_peered":"2024-09-16T00:39:00.051608+0000","last_unstale":"2024-09-16T00:40:59.584869+0000","last_undegraded":"2024-09-16T00:40:59.584869+0000","last_fullsized":"2024-09-16T00:40:59.584869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:58:56.320413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587926+0000","last_change":"2024-09-16T00:38:52.964571+0000","last_active":"2024-09-16T00:40:59.587926+0000","last_peered":"2024-09-16T00:40:59.587926+0000","last_clean":"2024-09-16T00:40:59.587926+0000","last_became_active":"2024-09-16T00:38:52.964234+0000","last_became_peered":"2024-09-16T00:38:52.964234+0000","last_unstale":"2024-09-16T00:40:59.587926+0000","last_undegraded":"2024-09-16T00:40:59.587926+0000","last_fullsized":"2024-09-16T00:40:59.587926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:22.782335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747773+0000","last_change":"2024-09-16T00:38:50.872185+0000","last_active":"2024-09-16T00:40:59.747773+0000","last_peered":"2024-09-16T00:40:59.747773+0000","last_clean":"2024-09-16T00:40:59.747773+0000","last_became_active":"2024-09-16T00:38:50.871496+0000","last_became_peered":"2024-09-16T00:38:50.871496+0000","last_unstale":"2024-09-16T00:40:59.747773+0000","last_undegraded":"2024-09-16T00:40:59.747773+0000","last_fullsized":"2024-09-16T00:40:59.747773+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:45.472341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747139+0000","last_change":"2024-09-16T00:39:00.044615+0000","last_active":"2024-09-16T00:40:59.747139+0000","last_peered":"2024-09-16T00:40:59.747139+0000","last_clean":"2024-09-16T00:40:59.747139+0000","last_became_active":"2024-09-16T00:39:00.044211+0000","last_became_peered":"2024-09-16T00:39:00.044211+0000","last_unstale":"2024-09-16T00:40:59.747139+0000","last_undegraded":"2024-09-16T00:40:59.747139+0000","last_fullsized":"2024-09-16T00:40:59.747139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:49.713756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161443+0000","last_change":"2024-09-16T00:39:00.045128+0000","last_active":"2024-09-16T00:41:00.161443+0000","last_peered":"2024-09-16T00:41:00.161443+0000","last_clean":"2024-09-16T00:41:00.161443+0000","last_became_active":"2024-09-16T00:39:00.044733+0000","last_became_peered":"2024-09-16T00:39:00.044733+0000","last_unstale":"2024-09-16T00:41:00.161443+0000","last_undegraded":"2024-09-16T00:41:00.161443+0000","last_fullsized":"2024-09-16T00:41:00.161443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:53.798740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.584969+0000","last_change":"2024-09-16T00:38:52.971265+0000","last_active":"2024-09-16T00:40:59.584969+0000","last_peered":"2024-09-16T00:40:59.584969+0000","last_clean":"2024-09-16T00:40:59.584969+0000","last_became_active":"2024-09-16T00:38:52.970947+0000","last_became_peered":"2024-09-16T00:38:52.970947+0000","last_unstale":"2024-09-16T00:40:59.584969+0000","last_undegraded":"2024-09-16T00:40:59.584969+0000","last_fullsized":"2024-09-16T00:40:59.584969+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:51.424115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588728+0000","last_change":"2024-09-16T00:38:50.873218+0000","last_active":"2024-09-16T00:40:59.588728+0000","last_peered":"2024-09-16T00:40:59.588728+0000","last_clean":"2024-09-16T00:40:59.588728+0000","last_became_active":"2024-09-16T00:38:50.872882+0000","last_became_peered":"2024-09-16T00:38:50.872882+0000","last_unstale":"2024-09-16T00:40:59.588728+0000","last_undegraded":"2024-09-16T00:40:59.588728+0000","last_fullsized":"2024-09-16T00:40:59.588728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:28.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747585+0000","last_change":"2024-09-16T00:39:00.056435+0000","last_active":"2024-09-16T00:40:59.747585+0000","last_peered":"2024-09-16T00:40:59.747585+0000","last_clean":"2024-09-16T00:40:59.747585+0000","last_became_active":"2024-09-16T00:39:00.056026+0000","last_became_peered":"2024-09-16T00:39:00.056026+0000","last_unstale":"2024-09-16T00:40:59.747585+0000","last_undegraded":"2024-09-16T00:40:59.747585+0000","last_fullsized":"2024-09-16T00:40:59.747585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:49.936430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747194+0000","last_change":"2024-09-16T00:38:52.957130+0000","last_active":"2024-09-16T00:40:59.747194+0000","last_peered":"2024-09-16T00:40:59.747194+0000","last_clean":"2024-09-16T00:40:59.747194+0000","last_became_active":"2024-09-16T00:38:52.956678+0000","last_became_peered":"2024-09-16T00:38:52.956678+0000","last_unstale":"2024-09-16T00:40:59.747194+0000","last_undegraded":"2024-09-16T00:40:59.747194+0000","last_fullsized":"2024-09-16T00:40:59.747194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:09.830626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.594858+0000","last_change":"2024-09-16T00:38:50.868227+0000","last_active":"2024-09-16T00:40:59.594858+0000","last_peered":"2024-09-16T00:40:59.594858+0000","last_clean":"2024-09-16T00:40:59.594858+0000","last_became_active":"2024-09-16T00:38:50.867863+0000","last_became_peered":"2024-09-16T00:38:50.867863+0000","last_unstale":"2024-09-16T00:40:59.594858+0000","last_undegraded":"2024-09-16T00:40:59.594858+0000","last_fullsized":"2024-09-16T00:40:59.594858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.620853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747376+0000","last_change":"2024-09-16T00:38:50.870899+0000","last_active":"2024-09-16T00:40:59.747376+0000","last_peered":"2024-09-16T00:40:59.747376+0000","last_clean":"2024-09-16T00:40:59.747376+0000","last_became_active":"2024-09-16T00:38:50.870531+0000","last_became_peered":"2024-09-16T00:38:50.870531+0000","last_unstale":"2024-09-16T00:40:59.747376+0000","last_undegraded":"2024-09-16T00:40:59.747376+0000","last_fullsized":"2024-09-16T00:40:59.747376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:14.108215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588250+0000","last_change":"2024-09-16T00:38:52.969135+0000","last_active":"2024-09-16T00:40:59.588250+0000","last_peered":"2024-09-16T00:40:59.588250+0000","last_clean":"2024-09-16T00:40:59.588250+0000","last_became_active":"2024-09-16T00:38:52.968265+0000","last_became_peered":"2024-09-16T00:38:52.968265+0000","last_unstale":"2024-09-16T00:40:59.588250+0000","last_undegraded":"2024-09-16T00:40:59.588250+0000","last_fullsized":"2024-09-16T00:40:59.588250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:58.412935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587684+0000","last_change":"2024-09-16T00:39:00.055356+0000","last_active":"2024-09-16T00:40:59.587684+0000","last_peered":"2024-09-16T00:40:59.587684+0000","last_clean":"2024-09-16T00:40:59.587684+0000","last_became_active":"2024-09-16T00:39:00.054945+0000","last_became_peered":"2024-09-16T00:39:00.054945+0000","last_unstale":"2024-09-16T00:40:59.587684+0000","last_undegraded":"2024-09-16T00:40:59.587684+0000","last_fullsized":"2024-09-16T00:40:59.587684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:53.284491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588368+0000","last_change":"2024-09-16T00:38:50.876752+0000","last_active":"2024-09-16T00:40:59.588368+0000","last_peered":"2024-09-16T00:40:59.588368+0000","last_clean":"2024-09-16T00:40:59.588368+0000","last_became_active":"2024-09-16T00:38:50.876437+0000","last_became_peered":"2024-09-16T00:38:50.876437+0000","last_unstale":"2024-09-16T00:40:59.588368+0000","last_undegraded":"2024-09-16T00:40:59.588368+0000","last_fullsized":"2024-09-16T00:40:59.588368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:49.856850+0000","last_clean_scrub_stamp":"2024-09-16T00:38:49.856850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.742197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586131+0000","last_change":"2024-09-16T00:38:52.969932+0000","last_active":"2024-09-16T00:40:59.586131+0000","last_peered":"2024-09-16T00:40:59.586131+0000","last_clean":"2024-09-16T00:40:59.586131+0000","last_became_active":"2024-09-16T00:38:52.969617+0000","last_became_peered":"2024-09-16T00:38:52.969617+0000","last_unstale":"2024-09-16T00:40:59.586131+0000","last_undegraded":"2024-09-16T00:40:59.586131+0000","last_fullsized":"2024-09-16T00:40:59.586131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:42.727392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588144+0000","last_change":"2024-09-16T00:39:00.055691+0000","last_active":"2024-09-16T00:40:59.588144+0000","last_peered":"2024-09-16T00:40:59.588144+0000","last_clean":"2024-09-16T00:40:59.588144+0000","last_became_active":"2024-09-16T00:39:00.054696+0000","last_became_peered":"2024-09-16T00:39:00.054696+0000","last_unstale":"2024-09-16T00:40:59.588144+0000","last_undegraded":"2024-09-16T00:40:59.588144+0000","last_fullsized":"2024-09-16T00:40:59.588144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:59.020554+0000","last_clean_scrub_stamp":"2024-09-16T00:38:59.020554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:08.506898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747412+0000","last_change":"2024-09-16T00:38:52.978590+0000","last_active":"2024-09-16T00:40:59.747412+0000","last_peered":"2024-09-16T00:40:59.747412+0000","last_clean":"2024-09-16T00:40:59.747412+0000","last_became_active":"2024-09-16T00:38:52.978246+0000","last_became_peered":"2024-09-16T00:38:52.978246+0000","last_unstale":"2024-09-16T00:40:59.747412+0000","last_undegraded":"2024-09-16T00:40:59.747412+0000","last_fullsized":"2024-09-16T00:40:59.747412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:43.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588676+0000","last_change":"2024-09-16T00:38:52.971717+0000","last_active":"2024-09-16T00:40:59.588676+0000","last_peered":"2024-09-16T00:40:59.588676+0000","last_clean":"2024-09-16T00:40:59.588676+0000","last_became_active":"2024-09-16T00:38:52.971363+0000","last_became_peered":"2024-09-16T00:38:52.971363+0000","last_unstale":"2024-09-16T00:40:59.588676+0000","last_undegraded":"2024-09-16T00:40:59.588676+0000","last_fullsized":"2024-09-16T00:40:59.588676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:29.438396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587600+0000","last_change":"2024-09-16T00:38:52.964757+0000","last_active":"2024-09-16T00:40:59.587600+0000","last_peered":"2024-09-16T00:40:59.587600+0000","last_clean":"2024-09-16T00:40:59.587600+0000","last_became_active":"2024-09-16T00:38:52.964449+0000","last_became_peered":"2024-09-16T00:38:52.964449+0000","last_unstale":"2024-09-16T00:40:59.587600+0000","last_undegraded":"2024-09-16T00:40:59.587600+0000","last_fullsized":"2024-09-16T00:40:59.587600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:23.568518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162004+0000","last_change":"2024-09-16T00:38:52.970469+0000","last_active":"2024-09-16T00:41:00.162004+0000","last_peered":"2024-09-16T00:41:00.162004+0000","last_clean":"2024-09-16T00:41:00.162004+0000","last_became_active":"2024-09-16T00:38:52.970158+0000","last_became_peered":"2024-09-16T00:38:52.970158+0000","last_unstale":"2024-09-16T00:41:00.162004+0000","last_undegraded":"2024-09-16T00:41:00.162004+0000","last_fullsized":"2024-09-16T00:41:00.162004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:59.956750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595476+0000","last_change":"2024-09-16T00:38:52.963358+0000","last_active":"2024-09-16T00:40:59.595476+0000","last_peered":"2024-09-16T00:40:59.595476+0000","last_clean":"2024-09-16T00:40:59.595476+0000","last_became_active":"2024-09-16T00:38:52.963014+0000","last_became_peered":"2024-09-16T00:38:52.963014+0000","last_unstale":"2024-09-16T00:40:59.595476+0000","last_undegraded":"2024-09-16T00:40:59.595476+0000","last_fullsized":"2024-09-16T00:40:59.595476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:34.396275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747786+0000","last_change":"2024-09-16T00:38:52.969736+0000","last_active":"2024-09-16T00:40:59.747786+0000","last_peered":"2024-09-16T00:40:59.747786+0000","last_clean":"2024-09-16T00:40:59.747786+0000","last_became_active":"2024-09-16T00:38:52.969412+0000","last_became_peered":"2024-09-16T00:38:52.969412+0000","last_unstale":"2024-09-16T00:40:59.747786+0000","last_undegraded":"2024-09-16T00:40:59.747786+0000","last_fullsized":"2024-09-16T00:40:59.747786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:41.433195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588960+0000","last_change":"2024-09-16T00:38:52.967061+0000","last_active":"2024-09-16T00:40:59.588960+0000","last_peered":"2024-09-16T00:40:59.588960+0000","last_clean":"2024-09-16T00:40:59.588960+0000","last_became_active":"2024-09-16T00:38:52.966604+0000","last_became_peered":"2024-09-16T00:38:52.966604+0000","last_unstale":"2024-09-16T00:40:59.588960+0000","last_undegraded":"2024-09-16T00:40:59.588960+0000","last_fullsized":"2024-09-16T00:40:59.588960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:20.179267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.748038+0000","last_change":"2024-09-16T00:38:52.965958+0000","last_active":"2024-09-16T00:40:59.748038+0000","last_peered":"2024-09-16T00:40:59.748038+0000","last_clean":"2024-09-16T00:40:59.748038+0000","last_became_active":"2024-09-16T00:38:52.965613+0000","last_became_peered":"2024-09-16T00:38:52.965613+0000","last_unstale":"2024-09-16T00:40:59.748038+0000","last_undegraded":"2024-09-16T00:40:59.748038+0000","last_fullsized":"2024-09-16T00:40:59.748038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:59.751698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587358+0000","last_change":"2024-09-16T00:38:52.969645+0000","last_active":"2024-09-16T00:40:59.587358+0000","last_peered":"2024-09-16T00:40:59.587358+0000","last_clean":"2024-09-16T00:40:59.587358+0000","last_became_active":"2024-09-16T00:38:52.968376+0000","last_became_peered":"2024-09-16T00:38:52.968376+0000","last_unstale":"2024-09-16T00:40:59.587358+0000","last_undegraded":"2024-09-16T00:40:59.587358+0000","last_fullsized":"2024-09-16T00:40:59.587358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.586993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747301+0000","last_change":"2024-09-16T00:38:52.969121+0000","last_active":"2024-09-16T00:40:59.747301+0000","last_peered":"2024-09-16T00:40:59.747301+0000","last_clean":"2024-09-16T00:40:59.747301+0000","last_became_active":"2024-09-16T00:38:52.968819+0000","last_became_peered":"2024-09-16T00:38:52.968819+0000","last_unstale":"2024-09-16T00:40:59.747301+0000","last_undegraded":"2024-09-16T00:40:59.747301+0000","last_fullsized":"2024-09-16T00:40:59.747301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:13.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587542+0000","last_change":"2024-09-16T00:38:52.972975+0000","last_active":"2024-09-16T00:40:59.587542+0000","last_peered":"2024-09-16T00:40:59.587542+0000","last_clean":"2024-09-16T00:40:59.587542+0000","last_became_active":"2024-09-16T00:38:52.972651+0000","last_became_peered":"2024-09-16T00:38:52.972651+0000","last_unstale":"2024-09-16T00:40:59.587542+0000","last_undegraded":"2024-09-16T00:40:59.587542+0000","last_fullsized":"2024-09-16T00:40:59.587542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:58.448018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595665+0000","last_change":"2024-09-16T00:38:52.964816+0000","last_active":"2024-09-16T00:40:59.595665+0000","last_peered":"2024-09-16T00:40:59.595665+0000","last_clean":"2024-09-16T00:40:59.595665+0000","last_became_active":"2024-09-16T00:38:52.964506+0000","last_became_peered":"2024-09-16T00:38:52.964506+0000","last_unstale":"2024-09-16T00:40:59.595665+0000","last_undegraded":"2024-09-16T00:40:59.595665+0000","last_fullsized":"2024-09-16T00:40:59.595665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:35.405671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747529+0000","last_change":"2024-09-16T00:38:52.968639+0000","last_active":"2024-09-16T00:40:59.747529+0000","last_peered":"2024-09-16T00:40:59.747529+0000","last_clean":"2024-09-16T00:40:59.747529+0000","last_became_active":"2024-09-16T00:38:52.968278+0000","last_became_peered":"2024-09-16T00:38:52.968278+0000","last_unstale":"2024-09-16T00:40:59.747529+0000","last_undegraded":"2024-09-16T00:40:59.747529+0000","last_fullsized":"2024-09-16T00:40:59.747529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:03.409735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.588319+0000","last_change":"2024-09-16T00:38:52.963539+0000","last_active":"2024-09-16T00:40:59.588319+0000","last_peered":"2024-09-16T00:40:59.588319+0000","last_clean":"2024-09-16T00:40:59.588319+0000","last_became_active":"2024-09-16T00:38:52.962414+0000","last_became_peered":"2024-09-16T00:38:52.962414+0000","last_unstale":"2024-09-16T00:40:59.588319+0000","last_undegraded":"2024-09-16T00:40:59.588319+0000","last_fullsized":"2024-09-16T00:40:59.588319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:56.391315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586100+0000","last_change":"2024-09-16T00:38:52.973956+0000","last_active":"2024-09-16T00:40:59.586100+0000","last_peered":"2024-09-16T00:40:59.586100+0000","last_clean":"2024-09-16T00:40:59.586100+0000","last_became_active":"2024-09-16T00:38:52.973632+0000","last_became_peered":"2024-09-16T00:38:52.973632+0000","last_unstale":"2024-09-16T00:40:59.586100+0000","last_undegraded":"2024-09-16T00:40:59.586100+0000","last_fullsized":"2024-09-16T00:40:59.586100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:48.158764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747818+0000","last_change":"2024-09-16T00:38:52.967077+0000","last_active":"2024-09-16T00:40:59.747818+0000","last_peered":"2024-09-16T00:40:59.747818+0000","last_clean":"2024-09-16T00:40:59.747818+0000","last_became_active":"2024-09-16T00:38:52.966677+0000","last_became_peered":"2024-09-16T00:38:52.966677+0000","last_unstale":"2024-09-16T00:40:59.747818+0000","last_undegraded":"2024-09-16T00:40:59.747818+0000","last_fullsized":"2024-09-16T00:40:59.747818+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:49.353391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587259+0000","last_change":"2024-09-16T00:38:52.972100+0000","last_active":"2024-09-16T00:40:59.587259+0000","last_peered":"2024-09-16T00:40:59.587259+0000","last_clean":"2024-09-16T00:40:59.587259+0000","last_became_active":"2024-09-16T00:38:52.971774+0000","last_became_peered":"2024-09-16T00:38:52.971774+0000","last_unstale":"2024-09-16T00:40:59.587259+0000","last_undegraded":"2024-09-16T00:40:59.587259+0000","last_fullsized":"2024-09-16T00:40:59.587259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:29.226824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161683+0000","last_change":"2024-09-16T00:38:52.968308+0000","last_active":"2024-09-16T00:41:00.161683+0000","last_peered":"2024-09-16T00:41:00.161683+0000","last_clean":"2024-09-16T00:41:00.161683+0000","last_became_active":"2024-09-16T00:38:52.967982+0000","last_became_peered":"2024-09-16T00:38:52.967982+0000","last_unstale":"2024-09-16T00:41:00.161683+0000","last_undegraded":"2024-09-16T00:41:00.161683+0000","last_fullsized":"2024-09-16T00:41:00.161683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:24.380112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.161806+0000","last_change":"2024-09-16T00:38:52.969084+0000","last_active":"2024-09-16T00:41:00.161806+0000","last_peered":"2024-09-16T00:41:00.161806+0000","last_clean":"2024-09-16T00:41:00.161806+0000","last_became_active":"2024-09-16T00:38:52.968790+0000","last_became_peered":"2024-09-16T00:38:52.968790+0000","last_unstale":"2024-09-16T00:41:00.161806+0000","last_undegraded":"2024-09-16T00:41:00.161806+0000","last_fullsized":"2024-09-16T00:41:00.161806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:49.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.595267+0000","last_change":"2024-09-16T00:38:52.966546+0000","last_active":"2024-09-16T00:40:59.595267+0000","last_peered":"2024-09-16T00:40:59.595267+0000","last_clean":"2024-09-16T00:40:59.595267+0000","last_became_active":"2024-09-16T00:38:52.966226+0000","last_became_peered":"2024-09-16T00:38:52.966226+0000","last_unstale":"2024-09-16T00:40:59.595267+0000","last_undegraded":"2024-09-16T00:40:59.595267+0000","last_fullsized":"2024-09-16T00:40:59.595267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:55.713894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590612+0000","last_change":"2024-09-16T00:38:52.972358+0000","last_active":"2024-09-16T00:40:59.590612+0000","last_peered":"2024-09-16T00:40:59.590612+0000","last_clean":"2024-09-16T00:40:59.590612+0000","last_became_active":"2024-09-16T00:38:52.971642+0000","last_became_peered":"2024-09-16T00:38:52.971642+0000","last_unstale":"2024-09-16T00:40:59.590612+0000","last_undegraded":"2024-09-16T00:40:59.590612+0000","last_fullsized":"2024-09-16T00:40:59.590612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:03.779091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592840+0000","last_change":"2024-09-16T00:38:52.975302+0000","last_active":"2024-09-16T00:40:59.592840+0000","last_peered":"2024-09-16T00:40:59.592840+0000","last_clean":"2024-09-16T00:40:59.592840+0000","last_became_active":"2024-09-16T00:38:52.974965+0000","last_became_peered":"2024-09-16T00:38:52.974965+0000","last_unstale":"2024-09-16T00:40:59.592840+0000","last_undegraded":"2024-09-16T00:40:59.592840+0000","last_fullsized":"2024-09-16T00:40:59.592840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:10.920283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747793+0000","last_change":"2024-09-16T00:38:52.968962+0000","last_active":"2024-09-16T00:40:59.747793+0000","last_peered":"2024-09-16T00:40:59.747793+0000","last_clean":"2024-09-16T00:40:59.747793+0000","last_became_active":"2024-09-16T00:38:52.968595+0000","last_became_peered":"2024-09-16T00:38:52.968595+0000","last_unstale":"2024-09-16T00:40:59.747793+0000","last_undegraded":"2024-09-16T00:40:59.747793+0000","last_fullsized":"2024-09-16T00:40:59.747793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:22.536875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.587933+0000","last_change":"2024-09-16T00:38:52.968937+0000","last_active":"2024-09-16T00:40:59.587933+0000","last_peered":"2024-09-16T00:40:59.587933+0000","last_clean":"2024-09-16T00:40:59.587933+0000","last_became_active":"2024-09-16T00:38:52.968183+0000","last_became_peered":"2024-09-16T00:38:52.968183+0000","last_unstale":"2024-09-16T00:40:59.587933+0000","last_undegraded":"2024-09-16T00:40:59.587933+0000","last_fullsized":"2024-09-16T00:40:59.587933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:35.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747138+0000","last_change":"2024-09-16T00:38:52.978386+0000","last_active":"2024-09-16T00:40:59.747138+0000","last_peered":"2024-09-16T00:40:59.747138+0000","last_clean":"2024-09-16T00:40:59.747138+0000","last_became_active":"2024-09-16T00:38:52.978066+0000","last_became_peered":"2024-09-16T00:38:52.978066+0000","last_unstale":"2024-09-16T00:40:59.747138+0000","last_undegraded":"2024-09-16T00:40:59.747138+0000","last_fullsized":"2024-09-16T00:40:59.747138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:31.243587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.592328+0000","last_change":"2024-09-16T00:38:52.971503+0000","last_active":"2024-09-16T00:40:59.592328+0000","last_peered":"2024-09-16T00:40:59.592328+0000","last_clean":"2024-09-16T00:40:59.592328+0000","last_became_active":"2024-09-16T00:38:52.971162+0000","last_became_peered":"2024-09-16T00:38:52.971162+0000","last_unstale":"2024-09-16T00:40:59.592328+0000","last_undegraded":"2024-09-16T00:40:59.592328+0000","last_fullsized":"2024-09-16T00:40:59.592328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:55.424618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747364+0000","last_change":"2024-09-16T00:38:52.967565+0000","last_active":"2024-09-16T00:40:59.747364+0000","last_peered":"2024-09-16T00:40:59.747364+0000","last_clean":"2024-09-16T00:40:59.747364+0000","last_became_active":"2024-09-16T00:38:52.967179+0000","last_became_peered":"2024-09-16T00:38:52.967179+0000","last_unstale":"2024-09-16T00:40:59.747364+0000","last_undegraded":"2024-09-16T00:40:59.747364+0000","last_fullsized":"2024-09-16T00:40:59.747364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:09:34.083432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.586035+0000","last_change":"2024-09-16T00:38:52.969114+0000","last_active":"2024-09-16T00:40:59.586035+0000","last_peered":"2024-09-16T00:40:59.586035+0000","last_clean":"2024-09-16T00:40:59.586035+0000","last_became_active":"2024-09-16T00:38:52.968683+0000","last_became_peered":"2024-09-16T00:38:52.968683+0000","last_unstale":"2024-09-16T00:40:59.586035+0000","last_undegraded":"2024-09-16T00:40:59.586035+0000","last_fullsized":"2024-09-16T00:40:59.586035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:44.325088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.590395+0000","last_change":"2024-09-16T00:38:52.975569+0000","last_active":"2024-09-16T00:40:59.590395+0000","last_peered":"2024-09-16T00:40:59.590395+0000","last_clean":"2024-09-16T00:40:59.590395+0000","last_became_active":"2024-09-16T00:38:52.974309+0000","last_became_peered":"2024-09-16T00:38:52.974309+0000","last_unstale":"2024-09-16T00:40:59.590395+0000","last_undegraded":"2024-09-16T00:40:59.590395+0000","last_fullsized":"2024-09-16T00:40:59.590395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:14.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162106+0000","last_change":"2024-09-16T00:38:52.974272+0000","last_active":"2024-09-16T00:41:00.162106+0000","last_peered":"2024-09-16T00:41:00.162106+0000","last_clean":"2024-09-16T00:41:00.162106+0000","last_became_active":"2024-09-16T00:38:52.973988+0000","last_became_peered":"2024-09-16T00:38:52.973988+0000","last_unstale":"2024-09-16T00:41:00.162106+0000","last_undegraded":"2024-09-16T00:41:00.162106+0000","last_fullsized":"2024-09-16T00:41:00.162106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:33.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:41:00.162139+0000","last_change":"2024-09-16T00:38:52.971889+0000","last_active":"2024-09-16T00:41:00.162139+0000","last_peered":"2024-09-16T00:41:00.162139+0000","last_clean":"2024-09-16T00:41:00.162139+0000","last_became_active":"2024-09-16T00:38:52.971594+0000","last_became_peered":"2024-09-16T00:38:52.971594+0000","last_unstale":"2024-09-16T00:41:00.162139+0000","last_undegraded":"2024-09-16T00:41:00.162139+0000","last_fullsized":"2024-09-16T00:41:00.162139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:01.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T00:40:59.747779+0000","last_change":"2024-09-16T00:38:52.966770+0000","last_active":"2024-09-16T00:40:59.747779+0000","last_peered":"2024-09-16T00:40:59.747779+0000","last_clean":"2024-09-16T00:40:59.747779+0000","last_became_active":"2024-09-16T00:38:52.966253+0000","last_became_peered":"2024-09-16T00:38:52.966253+0000","last_unstale":"2024-09-16T00:40:59.747779+0000","last_undegraded":"2024-09-16T00:40:59.747779+0000","last_fullsized":"2024-09-16T00:40:59.747779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:38:51.945299+0000","last_clean_scrub_stamp":"2024-09-16T00:38:51.945299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:17.150526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70239,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":222,"num_read_kb":183,"num_write":94,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":210717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547243,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":518379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1720320,"data_stored":1178382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939700,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":518961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168634,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":518983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1638400,"data_stored":1113425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":514283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1642496,"data_stored":1108929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":518401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":514387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444121,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":583758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":583724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":514373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:41:19.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:19.662 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:19 smithi102 ceph-mon[35364]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:20.104 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:19 smithi031 ceph-mon[30956]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.185+0000 7fbb40a3b640 1 -- 172.21.15.31:0/1229936857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c102790 msgr2=0x7fbb3c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.185+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/1229936857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c102790 0x7fbb3c1067e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009900 tx=0x7fbb2402a410 comp rx=0 tx=0).stop 2024-09-16T00:41:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.186+0000 7fbb40a3b640 1 -- 172.21.15.31:0/1229936857 shutdown_connections 2024-09-16T00:41:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.186+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/1229936857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb3c106f10 0x7fbb3c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.186+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/1229936857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c102790 0x7fbb3c1067e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.186+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/1229936857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb3c101de0 0x7fbb3c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.186+0000 7fbb40a3b640 1 -- 172.21.15.31:0/1229936857 >> 172.21.15.31:0/1229936857 conn(0x7fbb3c0fd8c0 msgr2=0x7fbb3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.186+0000 7fbb40a3b640 1 -- 172.21.15.31:0/1229936857 shutdown_connections 2024-09-16T00:41:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.186+0000 7fbb40a3b640 1 -- 172.21.15.31:0/1229936857 wait complete. 2024-09-16T00:41:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb40a3b640 1 Processor -- start 2024-09-16T00:41:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb40a3b640 1 -- start start 2024-09-16T00:41:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb40a3b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb3c101de0 0x7fbb3c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb40a3b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb3c102790 0x7fbb3c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb40a3b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c106f10 0x7fbb3c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb40a3b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c111f70 con 0x7fbb3c106f10 2024-09-16T00:41:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb40a3b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c111df0 con 0x7fbb3c101de0 2024-09-16T00:41:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb40a3b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c1120f0 con 0x7fbb3c102790 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb3ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c106f10 0x7fbb3c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb3ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c106f10 0x7fbb3c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49276/0 (socket says 172.21.15.31:49276) 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb3ad9d640 1 -- 172.21.15.31:0/767566893 learned_addr learned my addr 172.21.15.31:0/767566893 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb3a59c640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb3c101de0 0x7fbb3c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb31d9b640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb3c102790 0x7fbb3c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb3ad9d640 1 -- 172.21.15.31:0/767566893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb3c102790 msgr2=0x7fbb3c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.187+0000 7fbb3ad9d640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb3c102790 0x7fbb3c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.188+0000 7fbb3ad9d640 1 -- 172.21.15.31:0/767566893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb3c101de0 msgr2=0x7fbb3c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.188+0000 7fbb3ad9d640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb3c101de0 0x7fbb3c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.188+0000 7fbb3ad9d640 1 -- 172.21.15.31:0/767566893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c19d400 con 0x7fbb3c106f10 2024-09-16T00:41:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.188+0000 7fbb3ad9d640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c106f10 0x7fbb3c19cd20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400eaa0 tx=0x7fbb3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:20.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.188+0000 7fbb33fff640 1 -- 172.21.15.31:0/767566893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400cdb0 con 0x7fbb3c106f10 2024-09-16T00:41:20.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.188+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c19d6f0 con 0x7fbb3c106f10 2024-09-16T00:41:20.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.188+0000 7fbb33fff640 1 -- 172.21.15.31:0/767566893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb34005020 con 0x7fbb3c106f10 2024-09-16T00:41:20.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.188+0000 7fbb33fff640 1 -- 172.21.15.31:0/767566893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb340116b0 con 0x7fbb3c106f10 2024-09-16T00:41:20.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.188+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c1a4fd0 con 0x7fbb3c106f10 2024-09-16T00:41:20.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.189+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c1021c0 con 0x7fbb3c106f10 2024-09-16T00:41:20.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.190+0000 7fbb33fff640 1 -- 172.21.15.31:0/767566893 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbb34005560 con 0x7fbb3c106f10 2024-09-16T00:41:20.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.191+0000 7fbb33fff640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbb280a3240 0x7fbb280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:20.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.191+0000 7fbb3a59c640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbb280a3240 0x7fbb280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:20.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.191+0000 7fbb33fff640 1 -- 172.21.15.31:0/767566893 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fbb340cfed0 con 0x7fbb3c106f10 2024-09-16T00:41:20.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.191+0000 7fbb3a59c640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbb280a3240 0x7fbb280a5700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00bbe0 tx=0x7fbb2c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:20.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.195+0000 7fbb33fff640 1 -- 172.21.15.31:0/767566893 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb34016030 con 0x7fbb3c106f10 2024-09-16T00:41:20.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:19 smithi081 ceph-mon[35539]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:41:20.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.407+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb3c02d7c0 con 0x7fbb3c106f10 2024-09-16T00:41:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.407+0000 7fbb33fff640 1 -- 172.21.15.31:0/767566893 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fbb3409c840 con 0x7fbb3c106f10 2024-09-16T00:41:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:20.408 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:41:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:41:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbb280a3240 msgr2=0x7fbb280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbb280a3240 0x7fbb280a5700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00bbe0 tx=0x7fbb2c00b390 comp rx=0 tx=0).stop 2024-09-16T00:41:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c106f10 msgr2=0x7fbb3c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c106f10 0x7fbb3c19cd20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400eaa0 tx=0x7fbb3400ef60 comp rx=0 tx=0).stop 2024-09-16T00:41:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 shutdown_connections 2024-09-16T00:41:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb3c106f10 0x7fbb3c19cd20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbb280a3240 0x7fbb280a5700 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb3c102790 0x7fbb3c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 --2- 172.21.15.31:0/767566893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb3c101de0 0x7fbb3c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 >> 172.21.15.31:0/767566893 conn(0x7fbb3c0fd8c0 msgr2=0x7fbb3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 shutdown_connections 2024-09-16T00:41:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:20.410+0000 7fbb40a3b640 1 -- 172.21.15.31:0/767566893 wait complete. 2024-09-16T00:41:21.033 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:20 smithi031 ceph-mon[30956]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:21.034 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:20 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/767566893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:21.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:20 smithi102 ceph-mon[35364]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:21.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:20 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/767566893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:21.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:21.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:20 smithi081 ceph-mon[35539]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:21.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:20 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/767566893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:21.323 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.834+0000 7fce7584a640 1 -- 172.21.15.31:0/4100952774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 msgr2=0x7fce70110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:21.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.834+0000 7fce7584a640 1 --2- 172.21.15.31:0/4100952774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 0x7fce70110ca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fce60009ca0 tx=0x7fce6002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.834+0000 7fce7584a640 1 -- 172.21.15.31:0/4100952774 shutdown_connections 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.834+0000 7fce7584a640 1 --2- 172.21.15.31:0/4100952774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 0x7fce70110ca0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.834+0000 7fce7584a640 1 --2- 172.21.15.31:0/4100952774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce701049a0 0x7fce701089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.834+0000 7fce7584a640 1 --2- 172.21.15.31:0/4100952774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce70103ff0 0x7fce701043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.834+0000 7fce7584a640 1 -- 172.21.15.31:0/4100952774 >> 172.21.15.31:0/4100952774 conn(0x7fce700ffa90 msgr2=0x7fce70101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.834+0000 7fce7584a640 1 -- 172.21.15.31:0/4100952774 shutdown_connections 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.834+0000 7fce7584a640 1 -- 172.21.15.31:0/4100952774 wait complete. 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.835+0000 7fce7584a640 1 Processor -- start 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.835+0000 7fce7584a640 1 -- start start 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.835+0000 7fce7584a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce70103ff0 0x7fce7019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.835+0000 7fce7584a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce701049a0 0x7fce7019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.835+0000 7fce7584a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 0x7fce7019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:21.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.835+0000 7fce7584a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fce70114150 con 0x7fce70109120 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.835+0000 7fce7584a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fce70113fd0 con 0x7fce701049a0 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.835+0000 7fce7584a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce701142d0 con 0x7fce70103ff0 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 0x7fce7019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 0x7fce7019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49288/0 (socket says 172.21.15.31:49288) 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6f7fe640 1 -- 172.21.15.31:0/128940735 learned_addr learned my addr 172.21.15.31:0/128940735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6effd640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce70103ff0 0x7fce7019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6e7fc640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce701049a0 0x7fce7019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6f7fe640 1 -- 172.21.15.31:0/128940735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce70103ff0 msgr2=0x7fce7019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6f7fe640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce70103ff0 0x7fce7019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6f7fe640 1 -- 172.21.15.31:0/128940735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce701049a0 msgr2=0x7fce7019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6f7fe640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce701049a0 0x7fce7019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6f7fe640 1 -- 172.21.15.31:0/128940735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7019f560 con 0x7fce70109120 2024-09-16T00:41:21.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce6f7fe640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 0x7fce7019ee80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce60031a20 tx=0x7fce60031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce74848640 1 -- 172.21.15.31:0/128940735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60003b70 con 0x7fce70109120 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce74848640 1 -- 172.21.15.31:0/128940735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce60003d10 con 0x7fce70109120 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7019f7f0 con 0x7fce70109120 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce74848640 1 -- 172.21.15.31:0/128940735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6002fc40 con 0x7fce70109120 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.836+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce701a7130 con 0x7fce70109120 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.837+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce701043d0 con 0x7fce70109120 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.839+0000 7fce74848640 1 -- 172.21.15.31:0/128940735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fce60004050 con 0x7fce70109120 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.839+0000 7fce74848640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fce5c0a32c0 0x7fce5c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.840+0000 7fce6effd640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fce5c0a32c0 0x7fce5c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.840+0000 7fce74848640 1 -- 172.21.15.31:0/128940735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fce600f0f80 con 0x7fce70109120 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.840+0000 7fce6effd640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fce5c0a32c0 0x7fce5c0a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fce68009bd0 tx=0x7fce68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:21.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:21.843+0000 7fce74848640 1 -- 172.21.15.31:0/128940735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce600bd980 con 0x7fce70109120 2024-09-16T00:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.054+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce701a7360 con 0x7fce70109120 2024-09-16T00:41:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.055+0000 7fce74848640 1 -- 172.21.15.31:0/128940735 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fce600bd280 con 0x7fce70109120 2024-09-16T00:41:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:22.055 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:41:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:41:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fce5c0a32c0 msgr2=0x7fce5c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fce5c0a32c0 0x7fce5c0a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fce68009bd0 tx=0x7fce68005f90 comp rx=0 tx=0).stop 2024-09-16T00:41:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 msgr2=0x7fce7019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 0x7fce7019ee80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce60031a20 tx=0x7fce60031a50 comp rx=0 tx=0).stop 2024-09-16T00:41:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 shutdown_connections 2024-09-16T00:41:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce70109120 0x7fce7019ee80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce701049a0 0x7fce7019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fce5c0a32c0 0x7fce5c0a5780 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 --2- 172.21.15.31:0/128940735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce70103ff0 0x7fce7019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 >> 172.21.15.31:0/128940735 conn(0x7fce700ffa90 msgr2=0x7fce70101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 shutdown_connections 2024-09-16T00:41:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:22.057+0000 7fce7584a640 1 -- 172.21.15.31:0/128940735 wait complete. 2024-09-16T00:41:22.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:41:22.958 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:23.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:22 smithi102 ceph-mon[35364]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T00:41:23.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:22 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/128940735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:23.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:22 smithi031 ceph-mon[30956]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T00:41:23.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:22 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/128940735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:23.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:22 smithi081 ceph-mon[35539]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-16T00:41:23.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:22 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/128940735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.481+0000 7fb446562640 1 -- 172.21.15.31:0/1959229873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440101de0 msgr2=0x7fb4401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.481+0000 7fb446562640 1 --2- 172.21.15.31:0/1959229873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440101de0 0x7fb4401021c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb430009900 tx=0x7fb43002a410 comp rx=0 tx=0).stop 2024-09-16T00:41:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.481+0000 7fb446562640 1 -- 172.21.15.31:0/1959229873 shutdown_connections 2024-09-16T00:41:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.481+0000 7fb446562640 1 --2- 172.21.15.31:0/1959229873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb440106f10 0x7fb44010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.481+0000 7fb446562640 1 --2- 172.21.15.31:0/1959229873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb440102790 0x7fb4401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.481+0000 7fb446562640 1 --2- 172.21.15.31:0/1959229873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440101de0 0x7fb4401021c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.481+0000 7fb446562640 1 -- 172.21.15.31:0/1959229873 >> 172.21.15.31:0/1959229873 conn(0x7fb4400fd8c0 msgr2=0x7fb4400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.481+0000 7fb446562640 1 -- 172.21.15.31:0/1959229873 shutdown_connections 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.481+0000 7fb446562640 1 -- 172.21.15.31:0/1959229873 wait complete. 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.482+0000 7fb446562640 1 Processor -- start 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.482+0000 7fb446562640 1 -- start start 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.482+0000 7fb446562640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb440101de0 0x7fb4400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.482+0000 7fb446562640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440102790 0x7fb440074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.482+0000 7fb446562640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb440106f10 0x7fb440074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.482+0000 7fb446562640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb440111f70 con 0x7fb440102790 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.482+0000 7fb446562640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb440111df0 con 0x7fb440106f10 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.482+0000 7fb446562640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb4401120f0 con 0x7fb440101de0 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.482+0000 7fb43f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440102790 0x7fb440074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440102790 0x7fb440074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49298/0 (socket says 172.21.15.31:49298) 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43f7fe640 1 -- 172.21.15.31:0/232398968 learned_addr learned my addr 172.21.15.31:0/232398968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43ffff640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb440101de0 0x7fb4400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb444aff640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb440106f10 0x7fb440074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43f7fe640 1 -- 172.21.15.31:0/232398968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb440101de0 msgr2=0x7fb4400744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43f7fe640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb440101de0 0x7fb4400744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43f7fe640 1 -- 172.21.15.31:0/232398968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb440106f10 msgr2=0x7fb440074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43f7fe640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb440106f10 0x7fb440074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43f7fe640 1 -- 172.21.15.31:0/232398968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4401a68f0 con 0x7fb440102790 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb444aff640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb440106f10 0x7fb440074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43f7fe640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440102790 0x7fb440074a30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb43400d770 tx=0x7fb43400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43d7fa640 1 -- 172.21.15.31:0/232398968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434017070 con 0x7fb440102790 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43d7fa640 1 -- 172.21.15.31:0/232398968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb434005500 con 0x7fb440102790 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb43d7fa640 1 -- 172.21.15.31:0/232398968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434004e00 con 0x7fb440102790 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4401a6be0 con 0x7fb440102790 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.483+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4401a70d0 con 0x7fb440102790 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.484+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4401021c0 con 0x7fb440102790 2024-09-16T00:41:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.486+0000 7fb43d7fa640 1 -- 172.21.15.31:0/232398968 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb434004fa0 con 0x7fb440102790 2024-09-16T00:41:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.486+0000 7fb43d7fa640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb4280a3240 0x7fb4280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.486+0000 7fb43ffff640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb4280a3240 0x7fb4280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.486+0000 7fb43d7fa640 1 -- 172.21.15.31:0/232398968 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb4340cf740 con 0x7fb440102790 2024-09-16T00:41:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.487+0000 7fb43ffff640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb4280a3240 0x7fb4280a5700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb430009230 tx=0x7fb4300058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.490+0000 7fb43d7fa640 1 -- 172.21.15.31:0/232398968 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb43409c0b0 con 0x7fb440102790 2024-09-16T00:41:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.667+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb44010a0f0 con 0x7fb440102790 2024-09-16T00:41:23.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.669+0000 7fb43d7fa640 1 -- 172.21.15.31:0/232398968 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24766 (secure 0 0 0) 0x7fb43409b9b0 con 0x7fb440102790 2024-09-16T00:41:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:23.670 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:40:59.575845+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:41:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb4280a3240 msgr2=0x7fb4280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb4280a3240 0x7fb4280a5700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb430009230 tx=0x7fb4300058d0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440102790 msgr2=0x7fb440074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440102790 0x7fb440074a30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb43400d770 tx=0x7fb43400dc30 comp rx=0 tx=0).stop 2024-09-16T00:41:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 shutdown_connections 2024-09-16T00:41:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb440106f10 0x7fb440074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb440102790 0x7fb440074a30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb4280a3240 0x7fb4280a5700 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 --2- 172.21.15.31:0/232398968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb440101de0 0x7fb4400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 >> 172.21.15.31:0/232398968 conn(0x7fb4400fd8c0 msgr2=0x7fb4400fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 shutdown_connections 2024-09-16T00:41:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:23.671+0000 7fb446562640 1 -- 172.21.15.31:0/232398968 wait complete. 2024-09-16T00:41:23.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:23 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/232398968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:24.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:23 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/232398968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:24.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:23 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/232398968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:41:24.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-16T00:41:24.573 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:24.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:24 smithi031 ceph-mon[30956]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-16T00:41:25.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:24 smithi102 ceph-mon[35364]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-16T00:41:25.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:24 smithi081 ceph-mon[35539]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-16T00:41:25.758 INFO:teuthology.orchestra.run.smithi031.stdout:{"run_dir":"/var/run/ceph"} 2024-09-16T00:41:25.759 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-16T00:41:25.759 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3393 2024-09-16T00:41:25.759 DEBUG:tasks.quiescer:Yielding 2024-09-16T00:41:25.759 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T00:41:25.779 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:41:25.779 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T00:41:25.779 INFO:tasks.workunit:timeout=3h 2024-09-16T00:41:25.779 INFO:tasks.workunit:cleanup=True 2024-09-16T00:41:25.780 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T00:41:25.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:25.798 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T00:41:25.798 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-16T00:41:25.798 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-16T00:41:25.798 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:41:25.799 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-16T00:41:25.799 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 00:39:42.983504311 +0000 2024-09-16T00:41:25.799 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 00:39:42.983504311 +0000 2024-09-16T00:41:25.799 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 00:41:04.212731402 +0000 2024-09-16T00:41:25.799 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 00:39:42.983504311 +0000 2024-09-16T00:41:25.799 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T00:41:25.799 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T00:41:25.868 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:41:25.924 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T00:41:26.045 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:26.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.574+0000 7fd12ace1640 1 -- 172.21.15.31:0/3208930297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 msgr2=0x7fd12410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.574+0000 7fd12ace1640 1 --2- 172.21.15.31:0/3208930297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 0x7fd12410df30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd114009900 tx=0x7fd11402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.574+0000 7fd12ace1640 1 -- 172.21.15.31:0/3208930297 shutdown_connections 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.574+0000 7fd12ace1640 1 --2- 172.21.15.31:0/3208930297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd12410e500 0x7fd124110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.574+0000 7fd12ace1640 1 --2- 172.21.15.31:0/3208930297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 0x7fd12410df30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.574+0000 7fd12ace1640 1 --2- 172.21.15.31:0/3208930297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd124100b20 0x7fd124100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.574+0000 7fd12ace1640 1 -- 172.21.15.31:0/3208930297 >> 172.21.15.31:0/3208930297 conn(0x7fd1240fc7d0 msgr2=0x7fd1240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.574+0000 7fd12ace1640 1 -- 172.21.15.31:0/3208930297 shutdown_connections 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.574+0000 7fd12ace1640 1 -- 172.21.15.31:0/3208930297 wait complete. 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.575+0000 7fd12ace1640 1 Processor -- start 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.575+0000 7fd12ace1640 1 -- start start 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd12ace1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd124100b20 0x7fd12419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:26.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd12ace1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 0x7fd12419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:26.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd12ace1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd12410e500 0x7fd12419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:26.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd12ace1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd124114270 con 0x7fd124101440 2024-09-16T00:41:26.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd12ace1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd1241140f0 con 0x7fd124100b20 2024-09-16T00:41:26.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd12ace1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd1241143f0 con 0x7fd12410e500 2024-09-16T00:41:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd11bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 0x7fd12419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd11bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 0x7fd12419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49316/0 (socket says 172.21.15.31:49316) 2024-09-16T00:41:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd11bfff640 1 -- 172.21.15.31:0/976843114 learned_addr learned my addr 172.21.15.31:0/976843114 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd12927e640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd12410e500 0x7fd12419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd128a7d640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd124100b20 0x7fd12419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd11bfff640 1 -- 172.21.15.31:0/976843114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd12410e500 msgr2=0x7fd12419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd11bfff640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd12410e500 0x7fd12419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd11bfff640 1 -- 172.21.15.31:0/976843114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd124100b20 msgr2=0x7fd12419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd11bfff640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd124100b20 0x7fd12419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd11bfff640 1 -- 172.21.15.31:0/976843114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12419f6d0 con 0x7fd124101440 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd128a7d640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd124100b20 0x7fd12419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd11bfff640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 0x7fd12419ac60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd114006010 tx=0x7fd1140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.576+0000 7fd119ffb640 1 -- 172.21.15.31:0/976843114 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11403f030 con 0x7fd124101440 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.577+0000 7fd119ffb640 1 -- 172.21.15.31:0/976843114 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd114003ce0 con 0x7fd124101440 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.577+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12419f960 con 0x7fd124101440 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.577+0000 7fd119ffb640 1 -- 172.21.15.31:0/976843114 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114035070 con 0x7fd124101440 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.577+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1241a72a0 con 0x7fd124101440 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.578+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd12406a850 con 0x7fd124101440 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.579+0000 7fd119ffb640 1 -- 172.21.15.31:0/976843114 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd114042030 con 0x7fd124101440 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.580+0000 7fd119ffb640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd10c0a32c0 0x7fd10c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.580+0000 7fd119ffb640 1 -- 172.21.15.31:0/976843114 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd1140f0540 con 0x7fd124101440 2024-09-16T00:41:26.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.580+0000 7fd128a7d640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd10c0a32c0 0x7fd10c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.581+0000 7fd128a7d640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd10c0a32c0 0x7fd10c0a5780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd1200058c0 tx=0x7fd120005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.583+0000 7fd119ffb640 1 -- 172.21.15.31:0/976843114 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1140bce50 con 0x7fd124101440 2024-09-16T00:41:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.794+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd124109600 con 0x7fd124101440 2024-09-16T00:41:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.799+0000 7fd119ffb640 1 -- 172.21.15.31:0/976843114 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fd1140bc750 con 0x7fd124101440 2024-09-16T00:41:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:26.799 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:41:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:41:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd10c0a32c0 msgr2=0x7fd10c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd10c0a32c0 0x7fd10c0a5780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd1200058c0 tx=0x7fd120005d30 comp rx=0 tx=0).stop 2024-09-16T00:41:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 msgr2=0x7fd12419ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 0x7fd12419ac60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd114006010 tx=0x7fd1140042f0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 shutdown_connections 2024-09-16T00:41:26.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd10c0a32c0 0x7fd10c0a5780 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd12410e500 0x7fd12419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd124101440 0x7fd12419ac60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 --2- 172.21.15.31:0/976843114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd124100b20 0x7fd12419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:26.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 >> 172.21.15.31:0/976843114 conn(0x7fd1240fc7d0 msgr2=0x7fd1240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:26.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 shutdown_connections 2024-09-16T00:41:26.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:26.801+0000 7fd12ace1640 1 -- 172.21.15.31:0/976843114 wait complete. 2024-09-16T00:41:27.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:26 smithi102 ceph-mon[35364]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-16T00:41:27.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:26 smithi031 ceph-mon[30956]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-16T00:41:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:26 smithi081 ceph-mon[35539]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-16T00:41:27.564 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:38:53.576781+0000', 'modified': '2024-09-16T00:39:35.954166+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6839/3021718993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3021718993}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3021718993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6839/2243721734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 2243721734}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 2243721734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482]} 2024-09-16T00:41:27.564 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T00:41:27.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:41:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:28.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:27 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/976843114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:28.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:27 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/976843114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:28.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:27 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/976843114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.418+0000 7f41acdd7640 1 -- 172.21.15.31:0/3158795094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 msgr2=0x7f41a81021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.418+0000 7f41acdd7640 1 --2- 172.21.15.31:0/3158795094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 0x7f41a81021f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4198009900 tx=0x7f419802a410 comp rx=0 tx=0).stop 2024-09-16T00:41:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.418+0000 7f41acdd7640 1 -- 172.21.15.31:0/3158795094 shutdown_connections 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.418+0000 7f41acdd7640 1 --2- 172.21.15.31:0/3158795094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41a8106f40 0x7f41a810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.418+0000 7f41acdd7640 1 --2- 172.21.15.31:0/3158795094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a81027c0 0x7f41a8106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.418+0000 7f41acdd7640 1 --2- 172.21.15.31:0/3158795094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 0x7f41a81021f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.418+0000 7f41acdd7640 1 -- 172.21.15.31:0/3158795094 >> 172.21.15.31:0/3158795094 conn(0x7f41a80fd8d0 msgr2=0x7f41a80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.418+0000 7f41acdd7640 1 -- 172.21.15.31:0/3158795094 shutdown_connections 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.418+0000 7f41acdd7640 1 -- 172.21.15.31:0/3158795094 wait complete. 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41acdd7640 1 Processor -- start 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41acdd7640 1 -- start start 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41acdd7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 0x7f41a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41acdd7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41a81027c0 0x7f41a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41acdd7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a8106f40 0x7f41a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41acdd7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8111fa0 con 0x7f41a8101e10 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41acdd7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8111e20 con 0x7f41a81027c0 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41acdd7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8112120 con 0x7f41a8106f40 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41a659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 0x7f41a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41a659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 0x7f41a80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47980/0 (socket says 172.21.15.31:47980) 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.419+0000 7f41a659c640 1 -- 172.21.15.31:0/512710928 learned_addr learned my addr 172.21.15.31:0/512710928 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a8106f40 0x7f41a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41a5d9b640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41a81027c0 0x7f41a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41a659c640 1 -- 172.21.15.31:0/512710928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a8106f40 msgr2=0x7f41a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41a659c640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a8106f40 0x7f41a8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41a659c640 1 -- 172.21.15.31:0/512710928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41a81027c0 msgr2=0x7f41a8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41a659c640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41a81027c0 0x7f41a8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41a659c640 1 -- 172.21.15.31:0/512710928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a81a6960 con 0x7f41a8101e10 2024-09-16T00:41:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41a659c640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 0x7f41a80744f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4198006010 tx=0x7f41980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f419f7fe640 1 -- 172.21.15.31:0/512710928 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419803f030 con 0x7f41a8101e10 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f419f7fe640 1 -- 172.21.15.31:0/512710928 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4198003ce0 con 0x7f41a8101e10 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f419f7fe640 1 -- 172.21.15.31:0/512710928 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198035070 con 0x7f41a8101e10 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41acdd7640 1 -- 172.21.15.31:0/512710928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a81a6bf0 con 0x7f41a8101e10 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.420+0000 7f41acdd7640 1 -- 172.21.15.31:0/512710928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a81a7080 con 0x7f41a8101e10 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.422+0000 7f419f7fe640 1 -- 172.21.15.31:0/512710928 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4198042030 con 0x7f41a8101e10 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41af1b0740 1 Processor -- start 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41af1b0740 1 -- start start 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bd87db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 0x562b4bd92670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bd94fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41af1b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562b4bcee090 con 0x562b4bd879d0 2024-09-16T00:41:28.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41af1b0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562b4bc95a70 con 0x562b4bd92bb0 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41af1b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562b4bcea050 con 0x562b4bd882f0 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41a6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bd87db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41a6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bd87db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47986/0 (socket says 172.21.15.31:47986) 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41a6d9d640 1 -- 172.21.15.31:0/2206051726 learned_addr learned my addr 172.21.15.31:0/2206051726 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.423+0000 7f41a659c640 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 0x562b4bd92670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f419f7fe640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41940a32c0 0x7f41940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a5d9b640 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bd94fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f419f7fe640 1 -- 172.21.15.31:0/512710928 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f41980f01e0 con 0x7f41a8101e10 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a659c640 1 -- 172.21.15.31:0/2206051726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 msgr2=0x562b4bd94fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a659c640 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bd94fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a659c640 1 -- 172.21.15.31:0/2206051726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 msgr2=0x562b4bd87db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a659c640 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bd87db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a659c640 1 -- 172.21.15.31:0/2206051726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4bca2130 con 0x562b4bd882f0 2024-09-16T00:41:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bd87db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41940a32c0 0x7f41940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a659c640 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 0x562b4bd92670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41980b8e80 tx=0x7f419802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca35da48026fd53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f419cff9640 1 -- 172.21.15.31:0/2206051726 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198049070 con 0x562b4bd882f0 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.424+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41940a32c0 0x7f41940a5780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005f40 tx=0x7f41a0014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f419cff9640 1 -- 172.21.15.31:0/2206051726 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419803b580 con 0x562b4bd882f0 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f419cff9640 1 -- 172.21.15.31:0/2206051726 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41980bcbb0 con 0x562b4bd882f0 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f41af1b0740 1 -- 172.21.15.31:0/2206051726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 msgr2=0x562b4bd92670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 0x562b4bd92670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41980b8e80 tx=0x7f419802a410 comp rx=0 tx=0).stop 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f41af1b0740 1 -- 172.21.15.31:0/2206051726 shutdown_connections 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bd94fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 0x562b4bd92670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2206051726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bd87db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f41af1b0740 1 -- 172.21.15.31:0/2206051726 >> 172.21.15.31:0/2206051726 conn(0x562b4bd83f60 msgr2=0x562b4bd84340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f41af1b0740 1 -- 172.21.15.31:0/2206051726 shutdown_connections 2024-09-16T00:41:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.425+0000 7f41af1b0740 1 -- 172.21.15.31:0/2206051726 wait complete. 2024-09-16T00:41:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41af1b0740 1 -- start start 2024-09-16T00:41:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41af1b0740 1 Processor -- start 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd879d0 0x562b4bd923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 0x562b4bd8f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd92bb0 0x562b4bd8fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41af1b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562b4bd95540 con 0x562b4bd92bb0 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41af1b0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562b4bca8f20 con 0x562b4bd879d0 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41af1b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562b4bd956c0 con 0x562b4bd882f0 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41af1b0740 10 client.0 _ll_register_callbacks cb 0x562b4bc72030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41a6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd92bb0 0x562b4bd8fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41a6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd92bb0 0x562b4bd8fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48000/0 (socket says 172.21.15.31:48000) 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41a6d9d640 1 -- 172.21.15.31:0/3537090119 learned_addr learned my addr 172.21.15.31:0/3537090119 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41a659c640 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd879d0 0x562b4bd923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41a5d9b640 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 0x562b4bd8f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41a6d9d640 1 -- 172.21.15.31:0/3537090119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 msgr2=0x562b4bd8f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.426+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 0x562b4bd8f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f41a6d9d640 1 -- 172.21.15.31:0/3537090119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd879d0 msgr2=0x562b4bd923f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd879d0 0x562b4bd923f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f41a6d9d640 1 -- 172.21.15.31:0/3537090119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4ba5f2b0 con 0x562b4bd92bb0 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd92bb0 0x562b4bd8fc80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41a0011a60 tx=0x7f41a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c89a3feaebbace server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f418dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f418dffb640 1 -- 172.21.15.31:0/3537090119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a002b060 con 0x562b4bd92bb0 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f418dffb640 1 -- 172.21.15.31:0/3537090119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f41a001bdc0 con 0x562b4bd92bb0 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f41af1b0740 1 -- 172.21.15.31:0/3537090119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b4bcfb600 con 0x562b4bd92bb0 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f418dffb640 1 -- 172.21.15.31:0/3537090119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f41a0032050 con 0x562b4bd92bb0 2024-09-16T00:41:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f41af1b0740 10 client.15291 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f41af1b0740 1 -- 172.21.15.31:0/3537090119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562b4bda0d90 con 0x562b4bd92bb0 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.427+0000 7f418dffb640 1 -- 172.21.15.31:0/3537090119 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562b4bcfb600 con 0x562b4bd92bb0 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f418dffb640 1 -- 172.21.15.31:0/3537090119 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f41a000f5f0 con 0x562b4bd92bb0 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f41af1b0740 10 client.15291 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f41af1b0740 10 client.15291 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f41af1b0740 20 client.15291 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f41af1b0740 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562b4bda3450 0x562b4bda48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f41af1b0740 4 client.15291 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f41af1b0740 1 -- 172.21.15.31:0/3537090119 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b4bcf5b10 con 0x562b4bda3450 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f41a659c640 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562b4bda3450 0x562b4bda48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f41a659c640 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562b4bda3450 0x562b4bda48c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4d5abc61ce8800 server_cookie=ff3884943232285b in_seq=0 out_seq=0 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.428+0000 7f418dffb640 10 client.15291 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:28.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f418dffb640 1 -- 172.21.15.31:0/3537090119 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b4bcf5b10 con 0x562b4bda3450 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f418dffb640 10 client.15291 handle_command_reply: tid=0 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 client.15291 shutdown 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 -- 172.21.15.31:0/3537090119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd92bb0 msgr2=0x562b4bd8fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd92bb0 0x562b4bd8fc80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41a0011a60 tx=0x7f41a0004a40 comp rx=0 tx=0).stop 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 -- 172.21.15.31:0/3537090119 shutdown_connections 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd92bb0 0x562b4bd8fc80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562b4bda3450 0x562b4bda48c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd882f0 0x562b4bd8f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f418dffb640 0 client.15291 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 --2- 172.21.15.31:0/3537090119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd879d0 0x562b4bd923f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 -- 172.21.15.31:0/3537090119 >> 172.21.15.31:0/3537090119 conn(0x562b4bd83f60 msgr2=0x562b4bd89c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 -- 172.21.15.31:0/3537090119 shutdown_connections 2024-09-16T00:41:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 1 -- 172.21.15.31:0/3537090119 wait complete. 2024-09-16T00:41:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.429+0000 7f41af1b0740 20 client.15291 trim_cache size 0 max 16384 2024-09-16T00:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41af1b0740 1 Processor -- start 2024-09-16T00:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41af1b0740 1 -- start start 2024-09-16T00:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd879d0 0x562b4bd8f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd882f0 0x562b4bd9e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bd8e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41af1b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562b4bda2c00 con 0x562b4bd882f0 2024-09-16T00:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41af1b0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562b4bda32d0 con 0x562b4bd92bb0 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41af1b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562b4bda39a0 con 0x562b4bd879d0 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd882f0 0x562b4bd9e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd882f0 0x562b4bd9e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48010/0 (socket says 172.21.15.31:48010) 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a5d9b640 1 -- 172.21.15.31:0/1574207421 learned_addr learned my addr 172.21.15.31:0/1574207421 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a659c640 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd879d0 0x562b4bd8f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bd8e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a5d9b640 1 -- 172.21.15.31:0/1574207421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd879d0 msgr2=0x562b4bd8f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a5d9b640 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd879d0 0x562b4bd8f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a5d9b640 1 -- 172.21.15.31:0/1574207421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 msgr2=0x562b4bd8e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a5d9b640 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bd8e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a5d9b640 1 -- 172.21.15.31:0/1574207421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4bd850f0 con 0x562b4bd882f0 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a659c640 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd879d0 0x562b4bd8f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.448+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bd8e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f41a5d9b640 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd882f0 0x562b4bd9e160 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f419000bc00 tx=0x7f4190012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc357e71072dd208 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f418ffff640 1 -- 172.21.15.31:0/1574207421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190020070 con 0x562b4bd882f0 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f418ffff640 1 -- 172.21.15.31:0/1574207421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419000bdb0 con 0x562b4bd882f0 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f418ffff640 1 -- 172.21.15.31:0/1574207421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190010870 con 0x562b4bd882f0 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f41af1b0740 1 -- 172.21.15.31:0/1574207421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd882f0 msgr2=0x562b4bd9e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f41af1b0740 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd882f0 0x562b4bd9e160 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f419000bc00 tx=0x7f4190012820 comp rx=0 tx=0).stop 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f41af1b0740 1 -- 172.21.15.31:0/1574207421 shutdown_connections 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f41af1b0740 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bd8e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f41af1b0740 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd882f0 0x562b4bd9e160 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f41af1b0740 1 --2- 172.21.15.31:0/1574207421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4bd879d0 0x562b4bd8f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.449+0000 7f41af1b0740 1 -- 172.21.15.31:0/1574207421 >> 172.21.15.31:0/1574207421 conn(0x562b4bd83f60 msgr2=0x562b4bd8f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 -- 172.21.15.31:0/1574207421 shutdown_connections 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 -- 172.21.15.31:0/1574207421 wait complete. 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 -- start start 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 Processor -- start 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bda80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bda8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4be09b40 0x562b4be0bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562b4bda4740 con 0x562b4bd879d0 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562b4bda4070 con 0x562b4bd92bb0 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562b4bda4e10 con 0x562b4be09b40 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41a659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bda80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41a659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bda80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48018/0 (socket says 172.21.15.31:48018) 2024-09-16T00:41:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41a659c640 1 -- 172.21.15.31:0/2439690549 learned_addr learned my addr 172.21.15.31:0/2439690549 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41af1b0740 10 client.0 _ll_register_callbacks cb 0x7f41a0009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.450+0000 7f41a5d9b640 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bda8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41a6d9d640 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4be09b40 0x562b4be0bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41a659c640 1 -- 172.21.15.31:0/2439690549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4be09b40 msgr2=0x562b4be0bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41a659c640 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4be09b40 0x562b4be0bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41a659c640 1 -- 172.21.15.31:0/2439690549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 msgr2=0x562b4bda8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41a659c640 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bda8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41a659c640 1 -- 172.21.15.31:0/2439690549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4be0c4c0 con 0x562b4bd879d0 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41a659c640 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bda80d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41980ba760 tx=0x7f41980f0460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcbabc349c816a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f418dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f418dffb640 1 -- 172.21.15.31:0/2439690549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198049040 con 0x562b4bd879d0 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f418dffb640 1 -- 172.21.15.31:0/2439690549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f41980bc710 con 0x562b4bd879d0 2024-09-16T00:41:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41af1b0740 1 -- 172.21.15.31:0/2439690549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f419000bdb0 con 0x562b4bd879d0 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f418dffb640 1 -- 172.21.15.31:0/2439690549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f41980bbff0 con 0x562b4bd879d0 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41af1b0740 10 client.15297 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.451+0000 7f41af1b0740 1 -- 172.21.15.31:0/2439690549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562b4be0d7c0 con 0x562b4bd879d0 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.452+0000 7f418dffb640 1 -- 172.21.15.31:0/2439690549 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419000bdb0 con 0x562b4bd879d0 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.452+0000 7f418dffb640 1 -- 172.21.15.31:0/2439690549 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f41980c4c90 con 0x562b4bd879d0 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.452+0000 7f41af1b0740 10 client.15297 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.452+0000 7f41af1b0740 10 client.15297 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.452+0000 7f41af1b0740 20 client.15297 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.452+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562b4be12250 0x562b4be10f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.452+0000 7f41af1b0740 4 client.15297 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.452+0000 7f41af1b0740 1 -- 172.21.15.31:0/2439690549 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "status"}) -- 0x562b4bc720e0 con 0x562b4be12250 2024-09-16T00:41:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.452+0000 7f41a5d9b640 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562b4be12250 0x562b4be10f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41a5d9b640 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562b4be12250 0x562b4be10f60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6676c6bc660957b0 server_cookie=daee7d6a53213158 in_seq=0 out_seq=0 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f418dffb640 10 client.15297 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f418dffb640 1 -- 172.21.15.31:0/2439690549 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562b4bc720e0 con 0x562b4be12250 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f418dffb640 10 client.15297 handle_command_reply: tid=0 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 client.15297 shutdown 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 -- 172.21.15.31:0/2439690549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 msgr2=0x562b4bda80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bda80d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41980ba760 tx=0x7f41980f0460 comp rx=0 tx=0).stop 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 -- 172.21.15.31:0/2439690549 shutdown_connections 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562b4be12250 0x562b4be10f60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b4be09b40 0x562b4be0bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b4bd92bb0 0x562b4bda8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 --2- 172.21.15.31:0/2439690549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562b4bd879d0 0x562b4bda80d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 -- 172.21.15.31:0/2439690549 >> 172.21.15.31:0/2439690549 conn(0x562b4bd83f60 msgr2=0x562b4bd8f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f418dffb640 0 client.15297 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 -- 172.21.15.31:0/2439690549 shutdown_connections 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 1 -- 172.21.15.31:0/2439690549 wait complete. 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.453+0000 7f41af1b0740 20 client.15297 trim_cache size 0 max 16384 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "ccb601ea-73c2-11ef-bceb-c7b262605968", 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24482, 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-16T00:41:28.466 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-16T00:41:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-16T00:41:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 154.44672293400001, 2024-09-16T00:41:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 20, 2024-09-16T00:41:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 91, 2024-09-16T00:41:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-09-16T00:41:28.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 165.39446008900001 2024-09-16T00:41:28.467 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 -- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41940a32c0 msgr2=0x7f41940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41940a32c0 0x7f41940a5780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005f40 tx=0x7f41a0014040 comp rx=0 tx=0).stop 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 -- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 msgr2=0x7f41a80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 0x7f41a80744f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4198006010 tx=0x7f41980042f0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 -- 172.21.15.31:0/512710928 shutdown_connections 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f41940a32c0 0x7f41940a5780 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a8106f40 0x7f41a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41a81027c0 0x7f41a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 --2- 172.21.15.31:0/512710928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41a8101e10 0x7f41a80744f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 -- 172.21.15.31:0/512710928 >> 172.21.15.31:0/512710928 conn(0x7f41a80fd8d0 msgr2=0x7f41a80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 -- 172.21.15.31:0/512710928 shutdown_connections 2024-09-16T00:41:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:28.454+0000 7f41acdd7640 1 -- 172.21.15.31:0/512710928 wait complete. 2024-09-16T00:41:29.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:28 smithi031 ceph-mon[30956]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:29.108 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-16T00:41:29.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:28 smithi081 ceph-mon[35539]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:29.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:28 smithi102 ceph-mon[35364]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:31.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:30 smithi031 ceph-mon[30956]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T00:41:31.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:30 smithi081 ceph-mon[35539]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T00:41:31.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:30 smithi102 ceph-mon[35364]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T00:41:33.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:32 smithi031 ceph-mon[30956]: pgmap v368: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:33.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:32 smithi081 ceph-mon[35539]: pgmap v368: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:33.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:32 smithi102 ceph-mon[35364]: pgmap v368: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:35.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:34 smithi031 ceph-mon[30956]: pgmap v369: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:34 smithi081 ceph-mon[35539]: pgmap v369: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:35.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:34 smithi102 ceph-mon[35364]: pgmap v369: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:36 smithi031 ceph-mon[30956]: pgmap v370: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:37.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:36 smithi081 ceph-mon[35539]: pgmap v370: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:37.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:36 smithi102 ceph-mon[35364]: pgmap v370: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:39.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:38 smithi031 ceph-mon[30956]: pgmap v371: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:39.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:38 smithi081 ceph-mon[35539]: pgmap v371: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:39.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:38 smithi102 ceph-mon[35364]: pgmap v371: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:41.101 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:40 smithi102 ceph-mon[35364]: pgmap v372: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:41.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:40 smithi031 ceph-mon[30956]: pgmap v372: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:41.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:40 smithi081 ceph-mon[35539]: pgmap v372: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:43.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:42 smithi031 ceph-mon[30956]: pgmap v373: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:43.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:42 smithi081 ceph-mon[35539]: pgmap v373: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:43.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:42 smithi102 ceph-mon[35364]: pgmap v373: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:44.109 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.2 2024-09-16T00:41:44.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.906+0000 7feeaab84640 1 -- 172.21.15.31:0/2353940685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 msgr2=0x7feea410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.906+0000 7feeaab84640 1 --2- 172.21.15.31:0/2353940685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 0x7feea410ec50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009cd0 tx=0x7fee9c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:41:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.906+0000 7feeaab84640 1 -- 172.21.15.31:0/2353940685 shutdown_connections 2024-09-16T00:41:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.906+0000 7feeaab84640 1 --2- 172.21.15.31:0/2353940685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 0x7feea410ec50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.906+0000 7feeaab84640 1 --2- 172.21.15.31:0/2353940685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feea4103760 0x7feea4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.906+0000 7feeaab84640 1 --2- 172.21.15.31:0/2353940685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feea4102db0 0x7feea4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.906+0000 7feeaab84640 1 -- 172.21.15.31:0/2353940685 >> 172.21.15.31:0/2353940685 conn(0x7feea40fc7d0 msgr2=0x7feea40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.906+0000 7feeaab84640 1 -- 172.21.15.31:0/2353940685 shutdown_connections 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.906+0000 7feeaab84640 1 -- 172.21.15.31:0/2353940685 wait complete. 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.907+0000 7feeaab84640 1 Processor -- start 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.907+0000 7feeaab84640 1 -- start start 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feeaab84640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feea4102db0 0x7feea4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feeaab84640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feea4103760 0x7feea4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feeaab84640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 0x7feea419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feeaab84640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feea4112100 con 0x7feea41082e0 2024-09-16T00:41:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feeaab84640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7feea4111f80 con 0x7feea4102db0 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feeaab84640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feea4112280 con 0x7feea4103760 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea9121640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 0x7feea419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea8920640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feea4102db0 0x7feea4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea9121640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 0x7feea419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36834/0 (socket says 172.21.15.31:36834) 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea9121640 1 -- 172.21.15.31:0/394686633 learned_addr learned my addr 172.21.15.31:0/394686633 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea3fff640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feea4103760 0x7feea4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea9121640 1 -- 172.21.15.31:0/394686633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feea4103760 msgr2=0x7feea4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea9121640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feea4103760 0x7feea4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea9121640 1 -- 172.21.15.31:0/394686633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feea4102db0 msgr2=0x7feea4198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea9121640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feea4102db0 0x7feea4198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea9121640 1 -- 172.21.15.31:0/394686633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea419d530 con 0x7feea41082e0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea8920640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feea4102db0 0x7feea4198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.908+0000 7feea9121640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 0x7feea419ce50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fee9c005e30 tx=0x7fee9c032b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.909+0000 7feea1ffb640 1 -- 172.21.15.31:0/394686633 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c004240 con 0x7feea41082e0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.909+0000 7feea1ffb640 1 -- 172.21.15.31:0/394686633 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9c003e20 con 0x7feea41082e0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.909+0000 7feea1ffb640 1 -- 172.21.15.31:0/394686633 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c03a400 con 0x7feea41082e0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.909+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea419d7c0 con 0x7feea41082e0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.909+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea41a5100 con 0x7feea41082e0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.910+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea4103190 con 0x7feea41082e0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.911+0000 7feea1ffb640 1 -- 172.21.15.31:0/394686633 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fee9c0306c0 con 0x7feea41082e0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.912+0000 7feea1ffb640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fee940a3240 0x7fee940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.912+0000 7feea1ffb640 1 -- 172.21.15.31:0/394686633 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fee9c02e030 con 0x7feea41082e0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.912+0000 7feea8920640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fee940a3240 0x7fee940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.913+0000 7feea8920640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fee940a3240 0x7fee940a5700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fee9800bfd0 tx=0x7fee9800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:44.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:44.917+0000 7feea1ffb640 1 -- 172.21.15.31:0/394686633 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee9c0bd6a0 con 0x7feea41082e0 2024-09-16T00:41:45.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:44 smithi031 ceph-mon[30956]: pgmap v374: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.173+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feea410a2c0 con 0x7feea41082e0 2024-09-16T00:41:45.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.174+0000 7feea1ffb640 1 -- 172.21.15.31:0/394686633 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fee9c0bcfa0 con 0x7feea41082e0 2024-09-16T00:41:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:45.174 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:41:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:41:45.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fee940a3240 msgr2=0x7fee940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fee940a3240 0x7fee940a5700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fee9800bfd0 tx=0x7fee9800b390 comp rx=0 tx=0).stop 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 msgr2=0x7feea419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 0x7feea419ce50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fee9c005e30 tx=0x7fee9c032b50 comp rx=0 tx=0).stop 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 shutdown_connections 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feea41082e0 0x7feea419ce50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fee940a3240 0x7fee940a5700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feea4103760 0x7feea4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 --2- 172.21.15.31:0/394686633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feea4102db0 0x7feea4198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 >> 172.21.15.31:0/394686633 conn(0x7feea40fc7d0 msgr2=0x7feea4106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 shutdown_connections 2024-09-16T00:41:45.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:45.176+0000 7feeaab84640 1 -- 172.21.15.31:0/394686633 wait complete. 2024-09-16T00:41:45.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:44 smithi081 ceph-mon[35539]: pgmap v374: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:45.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:44 smithi102 ceph-mon[35364]: pgmap v374: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:46.009 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0' 2024-09-16T00:41:46.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0 2024-09-16T00:41:46.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:45 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/394686633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:46.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:45 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/394686633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:46.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:41:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:46.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:45 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/394686633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.821+0000 7fb8eea45640 1 -- 172.21.15.31:0/3856348700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102ef0 msgr2=0x7fb8e810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.821+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3856348700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102ef0 0x7fb8e810da40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009ca0 tx=0x7fb8e002a950 comp rx=0 tx=0).stop 2024-09-16T00:41:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.821+0000 7fb8eea45640 1 -- 172.21.15.31:0/3856348700 shutdown_connections 2024-09-16T00:41:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.821+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3856348700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102ef0 0x7fb8e810da40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.821+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3856348700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8102570 0x7fb8e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.821+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3856348700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8100e20 0x7fb8e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.821+0000 7fb8eea45640 1 -- 172.21.15.31:0/3856348700 >> 172.21.15.31:0/3856348700 conn(0x7fb8e80fc7d0 msgr2=0x7fb8e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.821+0000 7fb8eea45640 1 -- 172.21.15.31:0/3856348700 shutdown_connections 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.821+0000 7fb8eea45640 1 -- 172.21.15.31:0/3856348700 wait complete. 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8eea45640 1 Processor -- start 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8eea45640 1 -- start start 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8eea45640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8100e20 0x7fb8e80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8eea45640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8102570 0x7fb8e80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8eea45640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8102ef0 0x7fb8e80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8eea45640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8110f80 con 0x7fb8e8100e20 2024-09-16T00:41:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8eea45640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8110e00 con 0x7fb8e8102ef0 2024-09-16T00:41:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8eea45640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8111100 con 0x7fb8e8102570 2024-09-16T00:41:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8100e20 0x7fb8e80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8100e20 0x7fb8e80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36856/0 (socket says 172.21.15.31:36856) 2024-09-16T00:41:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8e7fff640 1 -- 172.21.15.31:0/3491416928 learned_addr learned my addr 172.21.15.31:0/3491416928 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8102570 0x7fb8e80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.822+0000 7fb8ecfe2640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8102ef0 0x7fb8e80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e7fff640 1 -- 172.21.15.31:0/3491416928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8102570 msgr2=0x7fb8e80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8102570 0x7fb8e80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e7fff640 1 -- 172.21.15.31:0/3491416928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8102ef0 msgr2=0x7fb8e80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8102ef0 0x7fb8e80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e7fff640 1 -- 172.21.15.31:0/3491416928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e81a69a0 con 0x7fb8e8100e20 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8102570 0x7fb8e80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8100e20 0x7fb8e80fcc80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800d770 tx=0x7fb8d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e57fa640 1 -- 172.21.15.31:0/3491416928 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8017070 con 0x7fb8e8100e20 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8eea45640 1 -- 172.21.15.31:0/3491416928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e81a6c90 con 0x7fb8e8100e20 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e57fa640 1 -- 172.21.15.31:0/3491416928 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d8005500 con 0x7fb8e8100e20 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8e57fa640 1 -- 172.21.15.31:0/3491416928 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8004e00 con 0x7fb8e8100e20 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.823+0000 7fb8eea45640 1 -- 172.21.15.31:0/3491416928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81a7180 con 0x7fb8e8100e20 2024-09-16T00:41:46.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.825+0000 7fb8e57fa640 1 -- 172.21.15.31:0/3491416928 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb8d8004fa0 con 0x7fb8e8100e20 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.826+0000 7fb8e57fa640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8d00a3240 0x7fb8d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.826+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8d00a3240 0x7fb8d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.826+0000 7fb8e57fa640 1 -- 172.21.15.31:0/3491416928 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb8d80cf7f0 con 0x7fb8e8100e20 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8f0e1e740 1 Processor -- start 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8f0e1e740 1 -- start start 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8d00a3240 0x7fb8d00a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005e00 tx=0x7fb8dc005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 0x56231eb1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1b080 0x56231eb25400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb25940 0x56231eb27d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56231ea802a0 con 0x56231eb1b080 2024-09-16T00:41:46.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56231ea27b10 con 0x56231eb25940 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56231ea7c260 con 0x56231eb1a760 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1b080 0x56231eb25400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1b080 0x56231eb25400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36864/0 (socket says 172.21.15.31:36864) 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8e77fe640 1 -- 172.21.15.31:0/131871094 learned_addr learned my addr 172.21.15.31:0/131871094 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/131871094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 0x56231eb1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8ecfe2640 1 --2- 172.21.15.31:0/131871094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb25940 0x56231eb27d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8e77fe640 1 -- 172.21.15.31:0/131871094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 msgr2=0x56231eb1ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/131871094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 0x56231eb1ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8e77fe640 1 -- 172.21.15.31:0/131871094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb25940 msgr2=0x56231eb27d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.827+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/131871094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb25940 0x56231eb27d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.828+0000 7fb8e77fe640 1 -- 172.21.15.31:0/131871094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231e7f1b90 con 0x56231eb1b080 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.828+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/131871094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1b080 0x56231eb25400 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00fe50 tx=0x7fb8dc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c56efe92e0121d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.828+0000 7fb8ce7fc640 1 -- 172.21.15.31:0/131871094 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc004230 con 0x56231eb1b080 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.828+0000 7fb8ce7fc640 1 -- 172.21.15.31:0/131871094 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8dc00fe80 con 0x56231eb1b080 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.828+0000 7fb8ce7fc640 1 -- 172.21.15.31:0/131871094 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc01e7c0 con 0x56231eb1b080 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/131871094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1b080 msgr2=0x56231eb25400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/131871094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1b080 0x56231eb25400 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00fe50 tx=0x7fb8dc00bf90 comp rx=0 tx=0).stop 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/131871094 shutdown_connections 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/131871094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb25940 0x56231eb27d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/131871094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1b080 0x56231eb25400 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/131871094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 0x56231eb1ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/131871094 >> 172.21.15.31:0/131871094 conn(0x56231eb16cf0 msgr2=0x56231eb170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/131871094 shutdown_connections 2024-09-16T00:41:46.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/131871094 wait complete. 2024-09-16T00:41:46.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.829+0000 7fb8f0e1e740 1 -- start start 2024-09-16T00:41:46.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8f0e1e740 1 Processor -- start 2024-09-16T00:41:46.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1a760 0x56231eb227d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 0x56231eb22d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb25940 0x56231eb23250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56231eb28510 con 0x56231eb1a760 2024-09-16T00:41:46.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56231ea086e0 con 0x56231eb1b080 2024-09-16T00:41:46.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56231eb28690 con 0x56231eb25940 2024-09-16T00:41:46.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1a760 0x56231eb227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1a760 0x56231eb227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36872/0 (socket says 172.21.15.31:36872) 2024-09-16T00:41:46.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8e7fff640 1 -- 172.21.15.31:0/2011010173 learned_addr learned my addr 172.21.15.31:0/2011010173 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:46.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8f0e1e740 10 client.0 _ll_register_callbacks cb 0x56231ea2ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:46.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 0x56231eb22d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.830+0000 7fb8ecfe2640 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb25940 0x56231eb23250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.831+0000 7fb8e7fff640 1 -- 172.21.15.31:0/2011010173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb25940 msgr2=0x56231eb23250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.831+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb25940 0x56231eb23250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.831+0000 7fb8e7fff640 1 -- 172.21.15.31:0/2011010173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 msgr2=0x56231eb22d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.831+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 0x56231eb22d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.831+0000 7fb8e7fff640 1 -- 172.21.15.31:0/2011010173 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231eb31a10 con 0x56231eb1a760 2024-09-16T00:41:46.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.831+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 0x56231eb22d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.831+0000 7fb8ecfe2640 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb25940 0x56231eb23250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.831+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1a760 0x56231eb227d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8097ec0 tx=0x7fb8d8004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f714301a407746bd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.831+0000 7fb8c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.833+0000 7fb8c77fe640 1 -- 172.21.15.31:0/2011010173 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8020040 con 0x56231eb1a760 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8c77fe640 1 -- 172.21.15.31:0/2011010173 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb8d8009cc0 con 0x56231eb1a760 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2011010173 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56231ea8d9d0 con 0x56231eb1a760 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8c77fe640 1 -- 172.21.15.31:0/2011010173 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb8d809fe10 con 0x56231eb1a760 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8f0e1e740 10 client.15315 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2011010173 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56231eb34430 con 0x56231eb1a760 2024-09-16T00:41:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8c77fe640 1 -- 172.21.15.31:0/2011010173 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56231ea8d9d0 con 0x56231eb1a760 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8c77fe640 1 -- 172.21.15.31:0/2011010173 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb8d809aeb0 con 0x56231eb1a760 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8f0e1e740 10 client.15315 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8f0e1e740 10 client.15315 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8f0e1e740 20 client.15315 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56231eb358b0 0x56231eb37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8f0e1e740 4 client.15315 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.834+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2011010173 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56231ea87d20 con 0x56231eb358b0 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.835+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56231eb358b0 0x56231eb37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.835+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56231eb358b0 0x56231eb37ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2951196671ae3f8 server_cookie=7090c4301e3f0195 in_seq=0 out_seq=0 2024-09-16T00:41:46.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.835+0000 7fb8c77fe640 10 client.15315 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.836+0000 7fb8c77fe640 1 -- 172.21.15.31:0/2011010173 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56231ea87d20 con 0x56231eb358b0 2024-09-16T00:41:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.836+0000 7fb8c77fe640 10 client.15315 handle_command_reply: tid=0 2024-09-16T00:41:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.836+0000 7fb8f0e1e740 1 client.15315 shutdown 2024-09-16T00:41:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2011010173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1a760 msgr2=0x56231eb227d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1a760 0x56231eb227d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8097ec0 tx=0x7fb8d8004780 comp rx=0 tx=0).stop 2024-09-16T00:41:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2011010173 shutdown_connections 2024-09-16T00:41:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56231eb358b0 0x56231eb37ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8c77fe640 0 client.15315 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:46.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb25940 0x56231eb23250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 0x56231eb22d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2011010173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb1a760 0x56231eb227d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2011010173 >> 172.21.15.31:0/2011010173 conn(0x56231eb16cf0 msgr2=0x56231eb15e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2011010173 shutdown_connections 2024-09-16T00:41:46.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2011010173 wait complete. 2024-09-16T00:41:46.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.837+0000 7fb8f0e1e740 20 client.15315 trim_cache size 0 max 16384 2024-09-16T00:41:46.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.855+0000 7fb8f0e1e740 1 Processor -- start 2024-09-16T00:41:46.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.855+0000 7fb8f0e1e740 1 -- start start 2024-09-16T00:41:46.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 0x56231eb15d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 0x56231eb23070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb3d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81aee20 con 0x56231eb25940 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81b16a0 con 0x56231eb1b080 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81af490 con 0x56231eb1a760 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8ecfe2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb3d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8ecfe2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb3d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36888/0 (socket says 172.21.15.31:36888) 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8ecfe2640 1 -- 172.21.15.31:0/2423821851 learned_addr learned my addr 172.21.15.31:0/2423821851 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/2423821851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 0x56231eb23070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/2423821851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 0x56231eb15d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8ecfe2640 1 -- 172.21.15.31:0/2423821851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 msgr2=0x56231eb15d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8ecfe2640 1 --2- 172.21.15.31:0/2423821851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 0x56231eb15d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8ecfe2640 1 -- 172.21.15.31:0/2423821851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 msgr2=0x56231eb23070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8ecfe2640 1 --2- 172.21.15.31:0/2423821851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 0x56231eb23070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8ecfe2640 1 -- 172.21.15.31:0/2423821851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231eb1aeb0 con 0x56231eb25940 2024-09-16T00:41:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.856+0000 7fb8ecfe2640 1 --2- 172.21.15.31:0/2423821851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb3d4d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0030020 tx=0x7fb8e0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3399235a4acea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.857+0000 7fb8cdffb640 1 -- 172.21.15.31:0/2423821851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0034040 con 0x56231eb25940 2024-09-16T00:41:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.857+0000 7fb8cdffb640 1 -- 172.21.15.31:0/2423821851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e002fb30 con 0x56231eb25940 2024-09-16T00:41:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.857+0000 7fb8cdffb640 1 -- 172.21.15.31:0/2423821851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0004370 con 0x56231eb25940 2024-09-16T00:41:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.858+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2423821851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 msgr2=0x56231eb3d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.858+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2423821851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb3d4d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0030020 tx=0x7fb8e0009800 comp rx=0 tx=0).stop 2024-09-16T00:41:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.858+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2423821851 shutdown_connections 2024-09-16T00:41:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.858+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2423821851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb3d4d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.858+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2423821851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1b080 0x56231eb23070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.858+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/2423821851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1a760 0x56231eb15d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.858+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2423821851 >> 172.21.15.31:0/2423821851 conn(0x56231eb16cf0 msgr2=0x56231eb15650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.858+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2423821851 shutdown_connections 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.858+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/2423821851 wait complete. 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.859+0000 7fb8f0e1e740 1 -- start start 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.859+0000 7fb8f0e1e740 1 Processor -- start 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.859+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1a760 0x56231eb1f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.859+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1b080 0x56231eb1fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.859+0000 7fb8f0e1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb20340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.859+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81afb60 con 0x56231eb25940 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.859+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81b0fd0 con 0x56231eb1a760 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.859+0000 7fb8f0e1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81b0230 con 0x56231eb1b080 2024-09-16T00:41:46.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.859+0000 7fb8f0e1e740 10 client.0 _ll_register_callbacks cb 0x7fb8d809b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8ecfe2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb20340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8ecfe2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb20340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36898/0 (socket says 172.21.15.31:36898) 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8e7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1a760 0x56231eb1f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8ecfe2640 1 -- 172.21.15.31:0/240869497 learned_addr learned my addr 172.21.15.31:0/240869497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1b080 0x56231eb1fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8e7fff640 1 -- 172.21.15.31:0/240869497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1b080 msgr2=0x56231eb1fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1b080 0x56231eb1fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8e7fff640 1 -- 172.21.15.31:0/240869497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 msgr2=0x56231eb20340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb20340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8e7fff640 1 -- 172.21.15.31:0/240869497 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231eb9f550 con 0x56231eb1a760 2024-09-16T00:41:46.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8ecfe2640 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb20340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1a760 0x56231eb1f8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800d740 tx=0x7fb8d809c2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f9a7ec5918a146f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.860+0000 7fb8c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8c77fe640 1 -- 172.21.15.31:0/240869497 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8020070 con 0x56231eb1a760 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8c77fe640 1 -- 172.21.15.31:0/240869497 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb8d80980d0 con 0x56231eb1a760 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/240869497 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e002fb30 con 0x56231eb1a760 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8c77fe640 1 -- 172.21.15.31:0/240869497 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb8d80977b0 con 0x56231eb1a760 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8f0e1e740 10 client.25028 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/240869497 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56231eba0810 con 0x56231eb1a760 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8c77fe640 1 -- 172.21.15.31:0/240869497 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d807f070 con 0x56231eb1a760 2024-09-16T00:41:46.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8c77fe640 1 -- 172.21.15.31:0/240869497 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb8d809aeb0 con 0x56231eb1a760 2024-09-16T00:41:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8f0e1e740 10 client.25028 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8f0e1e740 10 client.25028 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:41:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8f0e1e740 20 client.25028 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.861+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56231eb358b0 0x56231eba3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.862+0000 7fb8f0e1e740 4 client.25028 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}] 2024-09-16T00:41:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.862+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/240869497 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}) -- 0x56231ea03ec0 con 0x56231eb358b0 2024-09-16T00:41:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.862+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56231eb358b0 0x56231eba3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.862+0000 7fb8e77fe640 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56231eb358b0 0x56231eba3040 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fcc8662b9e4986c server_cookie=9966dc27fa668475 in_seq=0 out_seq=0 2024-09-16T00:41:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.862+0000 7fb8c77fe640 10 client.25028 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:46.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.870+0000 7fb8c77fe640 1 -- 172.21.15.31:0/240869497 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x56231ea03ec0 con 0x56231eb358b0 2024-09-16T00:41:46.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.870+0000 7fb8c77fe640 10 client.25028 handle_command_reply: tid=0 2024-09-16T00:41:46.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.870+0000 7fb8f0e1e740 1 client.25028 shutdown 2024-09-16T00:41:46.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.870+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/240869497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1a760 msgr2=0x56231eb1f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.870+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1a760 0x56231eb1f8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800d740 tx=0x7fb8d809c2a0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/240869497 shutdown_connections 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56231eb25940 0x56231eb20340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56231eb358b0 0x56231eba3040 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56231eb1b080 0x56231eb1fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8f0e1e740 1 --2- 172.21.15.31:0/240869497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56231eb1a760 0x56231eb1f8c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8c77fe640 0 client.25028 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/240869497 >> 172.21.15.31:0/240869497 conn(0x56231eb16cf0 msgr2=0x56231eb15e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/240869497 shutdown_connections 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8f0e1e740 1 -- 172.21.15.31:0/240869497 wait complete. 2024-09-16T00:41:46.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.871+0000 7fb8f0e1e740 20 client.25028 trim_cache size 0 max 16384 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 2, 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "923bb43": { 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 2, 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:41:46.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 150.2, 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:41:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:41:46.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:41:46.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:41:46.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:41:46.876 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:41:46.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 -- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8d00a3240 msgr2=0x7fb8d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8d00a3240 0x7fb8d00a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005e00 tx=0x7fb8dc005c70 comp rx=0 tx=0).stop 2024-09-16T00:41:46.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 -- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8100e20 msgr2=0x7fb8e80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:46.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8100e20 0x7fb8e80fcc80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800d770 tx=0x7fb8d800dc30 comp rx=0 tx=0).stop 2024-09-16T00:41:46.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 -- 172.21.15.31:0/3491416928 shutdown_connections 2024-09-16T00:41:46.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8102ef0 0x7fb8e80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb8d00a3240 0x7fb8d00a5700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8102570 0x7fb8e80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 --2- 172.21.15.31:0/3491416928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8100e20 0x7fb8e80fcc80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:46.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.872+0000 7fb8eea45640 1 -- 172.21.15.31:0/3491416928 >> 172.21.15.31:0/3491416928 conn(0x7fb8e80fc7d0 msgr2=0x7fb8e806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:46.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.873+0000 7fb8eea45640 1 -- 172.21.15.31:0/3491416928 shutdown_connections 2024-09-16T00:41:46.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:46.873+0000 7fb8eea45640 1 -- 172.21.15.31:0/3491416928 wait complete. 2024-09-16T00:41:46.879 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:47.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:46 smithi031 ceph-mon[30956]: pgmap v375: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:41:47.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:46 smithi081 ceph-mon[35539]: pgmap v375: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:41:47.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:46 smithi102 ceph-mon[35364]: pgmap v375: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:41:47.913 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '923bb43', quiesce took 3.8 seconds. Will release after: 26.4 2024-09-16T00:41:48.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.658+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/945613967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 msgr2=0x7f3a14107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:48.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.658+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/945613967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 0x7f3a14107b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009930 tx=0x7f3a0402a420 comp rx=0 tx=0).stop 2024-09-16T00:41:48.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.658+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/945613967 shutdown_connections 2024-09-16T00:41:48.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.658+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/945613967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a141082a0 0x7f3a1410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.658+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/945613967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 0x7f3a14107b90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.658+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/945613967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a14102d30 0x7f3a14103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.658+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/945613967 >> 172.21.15.31:0/945613967 conn(0x7f3a140fc7d0 msgr2=0x7f3a140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.658+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/945613967 shutdown_connections 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.658+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/945613967 wait complete. 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1bfa4640 1 Processor -- start 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1bfa4640 1 -- start start 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1bfa4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a14102d30 0x7f3a141985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1bfa4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 0x7f3a14198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1bfa4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a141082a0 0x7f3a1419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1bfa4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141120a0 con 0x7f3a141036e0 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1bfa4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14111f20 con 0x7f3a141082a0 2024-09-16T00:41:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1bfa4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14112220 con 0x7f3a14102d30 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1953f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 0x7f3a14198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1953f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 0x7f3a14198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42418/0 (socket says 172.21.15.31:42418) 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a1953f640 1 -- 172.21.15.31:0/1858525958 learned_addr learned my addr 172.21.15.31:0/1858525958 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.659+0000 7f3a19d40640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a14102d30 0x7f3a141985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a1a541640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a141082a0 0x7f3a1419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a1953f640 1 -- 172.21.15.31:0/1858525958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a14102d30 msgr2=0x7f3a141985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a1953f640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a14102d30 0x7f3a141985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a1953f640 1 -- 172.21.15.31:0/1858525958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a141082a0 msgr2=0x7f3a1419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a1953f640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a141082a0 0x7f3a1419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a1953f640 1 -- 172.21.15.31:0/1858525958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1419d570 con 0x7f3a141036e0 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a1953f640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 0x7f3a14198b00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3a04005940 tx=0x7f3a040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a0affd640 1 -- 172.21.15.31:0/1858525958 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0403f030 con 0x7f3a141036e0 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a0affd640 1 -- 172.21.15.31:0/1858525958 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a04003ce0 con 0x7f3a141036e0 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a0affd640 1 -- 172.21.15.31:0/1858525958 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04035070 con 0x7f3a141036e0 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/1858525958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1419d800 con 0x7f3a141036e0 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.660+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/1858525958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141a5140 con 0x7f3a141036e0 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.663+0000 7f3a1e37d740 1 Processor -- start 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.663+0000 7f3a1e37d740 1 -- start start 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc62880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 0x5627dcc6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 0x5627dcc6fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5627dcbc82e0 con 0x5627dcc624a0 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5627dcb6fce0 con 0x5627dcc62dc0 2024-09-16T00:41:48.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5627dcbc42a0 con 0x5627dcc6d680 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1a541640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc62880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1a541640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc62880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42430/0 (socket says 172.21.15.31:42430) 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1a541640 1 -- 172.21.15.31:0/466075978 learned_addr learned my addr 172.21.15.31:0/466075978 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a19d40640 1 --2- 172.21.15.31:0/466075978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 0x5627dcc6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a0affd640 1 -- 172.21.15.31:0/1858525958 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3a04042030 con 0x7f3a141036e0 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1a541640 1 -- 172.21.15.31:0/466075978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 msgr2=0x5627dcc6fa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1a541640 1 --2- 172.21.15.31:0/466075978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 0x5627dcc6fa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1a541640 1 -- 172.21.15.31:0/466075978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 msgr2=0x5627dcc6d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1a541640 1 --2- 172.21.15.31:0/466075978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 0x5627dcc6d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1a541640 1 -- 172.21.15.31:0/466075978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627dcb54440 con 0x5627dcc624a0 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a19d40640 1 --2- 172.21.15.31:0/466075978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 0x5627dcc6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.664+0000 7f3a1a541640 1 --2- 172.21.15.31:0/466075978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc62880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009f70 tx=0x7f3a0c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431ec0792362d96c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.665+0000 7f39fffff640 1 -- 172.21.15.31:0/466075978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c016070 con 0x5627dcc624a0 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.665+0000 7f39fffff640 1 -- 172.21.15.31:0/466075978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c004d20 con 0x5627dcc624a0 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.665+0000 7f39fffff640 1 -- 172.21.15.31:0/466075978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c012910 con 0x5627dcc624a0 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.665+0000 7f3a0affd640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3a000a3240 0x7f3a000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.665+0000 7f3a19d40640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3a000a3240 0x7f3a000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.665+0000 7f3a0affd640 1 -- 172.21.15.31:0/1858525958 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f3a040f00c0 con 0x7f3a141036e0 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 -- 172.21.15.31:0/466075978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 msgr2=0x5627dcc62880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a19d40640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3a000a3240 0x7f3a000a5700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5627dcc5f890 tx=0x7f3a1000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/466075978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc62880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009f70 tx=0x7f3a0c010800 comp rx=0 tx=0).stop 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 -- 172.21.15.31:0/466075978 shutdown_connections 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/466075978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 0x5627dcc6fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/466075978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 0x5627dcc6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/466075978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc62880 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 -- 172.21.15.31:0/466075978 >> 172.21.15.31:0/466075978 conn(0x5627dcc5ea30 msgr2=0x5627dcc5ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 -- 172.21.15.31:0/466075978 shutdown_connections 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 -- 172.21.15.31:0/466075978 wait complete. 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 -- start start 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.666+0000 7f3a1e37d740 1 Processor -- start 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 0x5627dcc6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 0x5627dcc6a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5627dcc70250 con 0x5627dcc624a0 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5627dcb83050 con 0x5627dcc62dc0 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5627dcc703d0 con 0x5627dcc6d680 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1e37d740 10 client.0 _ll_register_callbacks cb 0x5627dcbc3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a19d40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.040 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:41:48.040 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:41:48.040 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:41:48.040 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a19d40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc6d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42432/0 (socket says 172.21.15.31:42432) 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a19d40640 1 -- 172.21.15.31:0/497628659 learned_addr learned my addr 172.21.15.31:0/497628659 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1a541640 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 0x5627dcc6a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1953f640 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 0x5627dcc6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1a541640 1 -- 172.21.15.31:0/497628659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 msgr2=0x5627dcc6a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1a541640 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 0x5627dcc6a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1a541640 1 -- 172.21.15.31:0/497628659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 msgr2=0x5627dcc6d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1a541640 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc6d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a1a541640 1 -- 172.21.15.31:0/497628659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627dc9393c0 con 0x5627dcc6d680 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.667+0000 7f3a19d40640 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.668+0000 7f3a1a541640 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 0x5627dcc6a950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0125f0 tx=0x7f3a0c004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5adf109be9fc04e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.668+0000 7f39fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.668+0000 7f39fd7fa640 1 -- 172.21.15.31:0/497628659 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c029060 con 0x5627dcc6d680 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.668+0000 7f3a1e37d740 1 -- 172.21.15.31:0/497628659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a0c004d20 con 0x5627dcc6d680 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.668+0000 7f39fd7fa640 1 -- 172.21.15.31:0/497628659 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f3a0c0050e0 con 0x5627dcc6d680 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.668+0000 7f39fd7fa640 1 -- 172.21.15.31:0/497628659 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3a0c030050 con 0x5627dcc6d680 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.668+0000 7f3a1e37d740 10 client.24982 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.668+0000 7f3a1e37d740 1 -- 172.21.15.31:0/497628659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5627dcc7bd40 con 0x5627dcc6d680 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.668+0000 7f39fd7fa640 1 -- 172.21.15.31:0/497628659 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c004d20 con 0x5627dcc6d680 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f39fd7fa640 1 -- 172.21.15.31:0/497628659 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f3a0c00f040 con 0x5627dcc6d680 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f3a1e37d740 10 client.24982 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f3a1e37d740 10 client.24982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f3a1e37d740 20 client.24982 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5627dcc7d150 0x5627dcc7f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f3a1e37d740 4 client.24982 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f3a1e37d740 1 -- 172.21.15.31:0/497628659 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627dcbd5850 con 0x5627dcc7d150 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f3a19d40640 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5627dcc7d150 0x5627dcc7f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f3a19d40640 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5627dcc7d150 0x5627dcc7f540 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b99d93eb4cb202 server_cookie=4b1fa033661ffd0b in_seq=0 out_seq=0 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.669+0000 7f39fd7fa640 10 client.24982 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.670+0000 7f39fd7fa640 1 -- 172.21.15.31:0/497628659 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627dcbd5850 con 0x5627dcc7d150 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.670+0000 7f39fd7fa640 10 client.24982 handle_command_reply: tid=0 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.670+0000 7f3a1e37d740 1 client.24982 shutdown 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.670+0000 7f3a1e37d740 1 -- 172.21.15.31:0/497628659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 msgr2=0x5627dcc6a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.670+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 0x5627dcc6a950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0125f0 tx=0x7f3a0c004c70 comp rx=0 tx=0).stop 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.670+0000 7f3a1e37d740 1 -- 172.21.15.31:0/497628659 shutdown_connections 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.670+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5627dcc7d150 0x5627dcc7f540 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.670+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc6d680 0x5627dcc6a950 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.671+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc62dc0 0x5627dcc6a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.671+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/497628659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc624a0 0x5627dcc6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.671+0000 7f39fd7fa640 0 client.24982 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.671+0000 7f3a1e37d740 1 -- 172.21.15.31:0/497628659 >> 172.21.15.31:0/497628659 conn(0x5627dcc5ea30 msgr2=0x5627dcc64760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.671+0000 7f3a1e37d740 1 -- 172.21.15.31:0/497628659 shutdown_connections 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.671+0000 7f3a1e37d740 1 -- 172.21.15.31:0/497628659 wait complete. 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.671+0000 7f3a1e37d740 20 client.24982 trim_cache size 0 max 16384 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.688+0000 7f3a1e37d740 1 Processor -- start 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.688+0000 7f3a1e37d740 1 -- start start 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.688+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 0x5627dcc79630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.688+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 0x5627dcc650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.688+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcc675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.688+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141acd90 con 0x5627dcc6d680 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.688+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141af610 con 0x5627dcc624a0 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.688+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141ad400 con 0x5627dcc62dc0 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.688+0000 7f3a1a541640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcc675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a1a541640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcc675c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42438/0 (socket says 172.21.15.31:42438) 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a1a541640 1 -- 172.21.15.31:0/1419753203 learned_addr learned my addr 172.21.15.31:0/1419753203 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a19d40640 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 0x5627dcc79630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a1953f640 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 0x5627dcc650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a1a541640 1 -- 172.21.15.31:0/1419753203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 msgr2=0x5627dcc650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a1a541640 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 0x5627dcc650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a1a541640 1 -- 172.21.15.31:0/1419753203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 msgr2=0x5627dcc79630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a1a541640 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 0x5627dcc79630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a1a541640 1 -- 172.21.15.31:0/1419753203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627dcc6d400 con 0x5627dcc6d680 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a19d40640 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 0x5627dcc79630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f3a1a541640 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcc675c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011010 tx=0x7f3a0c0246b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20530d21d7f5d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f39ff7fe640 1 -- 172.21.15.31:0/1419753203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c016070 con 0x5627dcc6d680 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f39ff7fe640 1 -- 172.21.15.31:0/1419753203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c004ce0 con 0x5627dcc6d680 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.689+0000 7f39ff7fe640 1 -- 172.21.15.31:0/1419753203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0128f0 con 0x5627dcc6d680 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 -- 172.21.15.31:0/1419753203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 msgr2=0x5627dcc675c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcc675c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011010 tx=0x7f3a0c0246b0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 -- 172.21.15.31:0/1419753203 shutdown_connections 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcc675c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 0x5627dcc650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/1419753203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 0x5627dcc79630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 -- 172.21.15.31:0/1419753203 >> 172.21.15.31:0/1419753203 conn(0x5627dcc5ea30 msgr2=0x5627dcc6c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:48.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 -- 172.21.15.31:0/1419753203 shutdown_connections 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 -- 172.21.15.31:0/1419753203 wait complete. 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.690+0000 7f3a1e37d740 1 -- start start 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1e37d740 1 Processor -- start 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 0x5627dcc815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 0x5627dcc81b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1e37d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcce5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141adad0 con 0x5627dcc6d680 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141aef40 con 0x5627dcc624a0 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1e37d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141ae1a0 con 0x5627dcc62dc0 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1e37d740 10 client.0 _ll_register_callbacks cb 0x7f3a0c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1a541640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcce5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1a541640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcce5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42446/0 (socket says 172.21.15.31:42446) 2024-09-16T00:41:48.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1a541640 1 -- 172.21.15.31:0/2316247678 learned_addr learned my addr 172.21.15.31:0/2316247678 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a1953f640 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 0x5627dcc81b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.691+0000 7f3a19d40640 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 0x5627dcc815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f3a1a541640 1 -- 172.21.15.31:0/2316247678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 msgr2=0x5627dcc81b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f3a1a541640 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 0x5627dcc81b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f3a1a541640 1 -- 172.21.15.31:0/2316247678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 msgr2=0x5627dcc815f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f3a1a541640 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 0x5627dcc815f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f3a1a541640 1 -- 172.21.15.31:0/2316247678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627dcce6020 con 0x5627dcc6d680 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f3a1a541640 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcce5a90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c010840 tx=0x7f3a0c024530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f0164cbad1f0a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f39fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f39fd7fa640 1 -- 172.21.15.31:0/2316247678 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c016040 con 0x5627dcc6d680 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f39fd7fa640 1 -- 172.21.15.31:0/2316247678 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f3a0c021070 con 0x5627dcc6d680 2024-09-16T00:41:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f3a1e37d740 1 -- 172.21.15.31:0/2316247678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a0c004ce0 con 0x5627dcc6d680 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f39fd7fa640 1 -- 172.21.15.31:0/2316247678 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3a0c012b40 con 0x5627dcc6d680 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f3a1e37d740 10 client.15339 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f3a1e37d740 1 -- 172.21.15.31:0/2316247678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5627dcce7300 con 0x5627dcc6d680 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.692+0000 7f39fd7fa640 1 -- 172.21.15.31:0/2316247678 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c004ce0 con 0x5627dcc6d680 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f39fd7fa640 1 -- 172.21.15.31:0/2316247678 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f3a0c0054b0 con 0x5627dcc6d680 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f3a1e37d740 10 client.15339 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f3a1e37d740 10 client.15339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f3a1e37d740 20 client.15339 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5627dcce7960 0x5627dcce9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f3a1e37d740 4 client.15339 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f3a1e37d740 1 -- 172.21.15.31:0/2316247678 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627dcbcfd60 con 0x5627dcce7960 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f3a19d40640 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5627dcce7960 0x5627dcce9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f3a19d40640 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5627dcce7960 0x5627dcce9d50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4140b7707a949fe7 server_cookie=fc74c4233a22b1db in_seq=0 out_seq=0 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f39fd7fa640 10 client.15339 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f39fd7fa640 1 -- 172.21.15.31:0/2316247678 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5627dcbcfd60 con 0x5627dcce7960 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f39fd7fa640 10 client.15339 handle_command_reply: tid=0 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.693+0000 7f3a1e37d740 1 client.15339 shutdown 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 -- 172.21.15.31:0/2316247678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 msgr2=0x5627dcce5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcce5a90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c010840 tx=0x7f3a0c024530 comp rx=0 tx=0).stop 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 -- 172.21.15.31:0/2316247678 shutdown_connections 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627dcc6d680 0x5627dcce5a90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5627dcce7960 0x5627dcce9d50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dcc62dc0 0x5627dcc81b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 --2- 172.21.15.31:0/2316247678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627dcc624a0 0x5627dcc815f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f39fd7fa640 0 client.15339 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 -- 172.21.15.31:0/2316247678 >> 172.21.15.31:0/2316247678 conn(0x5627dcc5ea30 msgr2=0x5627dcc5d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 -- 172.21.15.31:0/2316247678 shutdown_connections 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 1 -- 172.21.15.31:0/2316247678 wait complete. 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1e37d740 20 client.15339 trim_cache size 0 max 16384 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3a000a3240 msgr2=0x7f3a000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3a000a3240 0x7f3a000a5700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5627dcc5f890 tx=0x7f3a1000b390 comp rx=0 tx=0).stop 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 msgr2=0x7f3a14198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 0x7f3a14198b00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3a04005940 tx=0x7f3a040042f0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.694+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/1858525958 shutdown_connections 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.695+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a141082a0 0x7f3a1419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.695+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a141036e0 0x7f3a14198b00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.695+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f3a000a3240 0x7f3a000a5700 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.695+0000 7f3a1bfa4640 1 --2- 172.21.15.31:0/1858525958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a14102d30 0x7f3a141985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.695+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/1858525958 >> 172.21.15.31:0/1858525958 conn(0x7f3a140fc7d0 msgr2=0x7f3a14105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:48.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.695+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/1858525958 shutdown_connections 2024-09-16T00:41:48.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:47.695+0000 7f3a1bfa4640 1 -- 172.21.15.31:0/1858525958 wait complete. 2024-09-16T00:41:48.677 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:41:48.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:49.075 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:48 smithi031 ceph-mon[30956]: pgmap v376: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:49.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:48 smithi081 ceph-mon[35539]: pgmap v376: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:49.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:48 smithi102 ceph-mon[35364]: pgmap v376: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:51.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.193+0000 7fa219510640 1 -- 172.21.15.31:0/3978414858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 msgr2=0x7fa21410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.193+0000 7fa219510640 1 --2- 172.21.15.31:0/3978414858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 0x7fa21410ea90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009ca0 tx=0x7fa20c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.193+0000 7fa219510640 1 -- 172.21.15.31:0/3978414858 shutdown_connections 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.193+0000 7fa219510640 1 --2- 172.21.15.31:0/3978414858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 0x7fa21410ea90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.193+0000 7fa219510640 1 --2- 172.21.15.31:0/3978414858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa214102790 0x7fa2141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.193+0000 7fa219510640 1 --2- 172.21.15.31:0/3978414858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa214101de0 0x7fa2141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.193+0000 7fa219510640 1 -- 172.21.15.31:0/3978414858 >> 172.21.15.31:0/3978414858 conn(0x7fa2140fd8e0 msgr2=0x7fa2140ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.193+0000 7fa219510640 1 -- 172.21.15.31:0/3978414858 shutdown_connections 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.193+0000 7fa219510640 1 -- 172.21.15.31:0/3978414858 wait complete. 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.194+0000 7fa219510640 1 Processor -- start 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.194+0000 7fa219510640 1 -- start start 2024-09-16T00:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa219510640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa214101de0 0x7fa2141983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa219510640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa214102790 0x7fa214198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa219510640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 0x7fa21419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa219510640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa214111f70 con 0x7fa214106f10 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa219510640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa214111df0 con 0x7fa214102790 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa219510640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa2141120f0 con 0x7fa214101de0 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa2137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 0x7fa21419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa214102790 0x7fa214198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa2137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 0x7fa21419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42458/0 (socket says 172.21.15.31:42458) 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa214102790 0x7fa214198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:44738/0 (socket says 172.21.15.31:44738) 2024-09-16T00:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa2137fe640 1 -- 172.21.15.31:0/3376024521 learned_addr learned my addr 172.21.15.31:0/3376024521 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:41:51.122 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:41:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa212ffd640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa214101de0 0x7fa2141983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa2137fe640 1 -- 172.21.15.31:0/3376024521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa214101de0 msgr2=0x7fa2141983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa2137fe640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa214101de0 0x7fa2141983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa2137fe640 1 -- 172.21.15.31:0/3376024521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa214102790 msgr2=0x7fa214198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa2137fe640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa214102790 0x7fa214198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa2137fe640 1 -- 172.21.15.31:0/3376024521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21419d3a0 con 0x7fa214106f10 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa20bfff640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa214102790 0x7fa214198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.195+0000 7fa2137fe640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 0x7fa21419ccc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa20c031a20 tx=0x7fa20c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.196+0000 7fa210ff9640 1 -- 172.21.15.31:0/3376024521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c003c30 con 0x7fa214106f10 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.196+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21419d630 con 0x7fa214106f10 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.196+0000 7fa210ff9640 1 -- 172.21.15.31:0/3376024521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20c003dd0 con 0x7fa214106f10 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.196+0000 7fa210ff9640 1 -- 172.21.15.31:0/3376024521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c02fcf0 con 0x7fa214106f10 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.196+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2141a4f70 con 0x7fa214106f10 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.197+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2141021c0 con 0x7fa214106f10 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.198+0000 7fa210ff9640 1 -- 172.21.15.31:0/3376024521 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa20c004050 con 0x7fa214106f10 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.199+0000 7fa210ff9640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa2000a3240 0x7fa2000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.199+0000 7fa210ff9640 1 -- 172.21.15.31:0/3376024521 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa20c0f1140 con 0x7fa214106f10 2024-09-16T00:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.199+0000 7fa212ffd640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa2000a3240 0x7fa2000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.199+0000 7fa212ffd640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa2000a3240 0x7fa2000a5700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa20400bbe0 tx=0x7fa20400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.202+0000 7fa210ff9640 1 -- 172.21.15.31:0/3376024521 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa20c0bdb40 con 0x7fa214106f10 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.425+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa214199750 con 0x7fa214106f10 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.426+0000 7fa210ff9640 1 -- 172.21.15.31:0/3376024521 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fa214199750 con 0x7fa214106f10 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa2000a3240 msgr2=0x7fa2000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa2000a3240 0x7fa2000a5700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa20400bbe0 tx=0x7fa20400b390 comp rx=0 tx=0).stop 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 msgr2=0x7fa21419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 0x7fa21419ccc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa20c031a20 tx=0x7fa20c031a50 comp rx=0 tx=0).stop 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 shutdown_connections 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa214106f10 0x7fa21419ccc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa214102790 0x7fa214198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa2000a3240 0x7fa2000a5700 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 --2- 172.21.15.31:0/3376024521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa214101de0 0x7fa2141983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 >> 172.21.15.31:0/3376024521 conn(0x7fa2140fd8e0 msgr2=0x7fa2140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 shutdown_connections 2024-09-16T00:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:50.428+0000 7fa219510640 1 -- 172.21.15.31:0/3376024521 wait complete. 2024-09-16T00:41:51.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:50 smithi031 ceph-mon[30956]: pgmap v377: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:51.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:50 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3376024521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:51.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:50 smithi081 ceph-mon[35539]: pgmap v377: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:51.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:50 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3376024521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:51.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:50 smithi102 ceph-mon[35364]: pgmap v377: 129 pgs: 129 active+clean; 651 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:41:51.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:50 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3376024521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:41:53.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:52 smithi031 ceph-mon[30956]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:53.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:41:53.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:52 smithi081 ceph-mon[35539]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:53.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:52 smithi102 ceph-mon[35364]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-16T00:41:53.437 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:53.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.956+0000 7f93c9f58640 1 -- 172.21.15.31:0/427912655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 msgr2=0x7f93c4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:53.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.956+0000 7f93c9f58640 1 --2- 172.21.15.31:0/427912655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 0x7f93c4101200 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009900 tx=0x7f93b402a480 comp rx=0 tx=0).stop 2024-09-16T00:41:53.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.956+0000 7f93c9f58640 1 -- 172.21.15.31:0/427912655 shutdown_connections 2024-09-16T00:41:53.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.956+0000 7f93c9f58640 1 --2- 172.21.15.31:0/427912655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c4102ef0 0x7f93c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.956+0000 7f93c9f58640 1 --2- 172.21.15.31:0/427912655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c4102570 0x7f93c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.956+0000 7f93c9f58640 1 --2- 172.21.15.31:0/427912655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 0x7f93c4101200 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.956+0000 7f93c9f58640 1 -- 172.21.15.31:0/427912655 >> 172.21.15.31:0/427912655 conn(0x7f93c40fc7d0 msgr2=0x7f93c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.957+0000 7f93c9f58640 1 -- 172.21.15.31:0/427912655 shutdown_connections 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.957+0000 7f93c9f58640 1 -- 172.21.15.31:0/427912655 wait complete. 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.957+0000 7f93c9f58640 1 Processor -- start 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.957+0000 7f93c9f58640 1 -- start start 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c9f58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 0x7f93c4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c9f58640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c4102570 0x7f93c41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c9f58640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c4102ef0 0x7f93c419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c9f58640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4110f80 con 0x7f93c4100e20 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c9f58640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4110e00 con 0x7f93c4102ef0 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c9f58640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4111100 con 0x7f93c4102570 2024-09-16T00:41:53.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 0x7f93c4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 0x7f93c4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42474/0 (socket says 172.21.15.31:42474) 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c37fe640 1 -- 172.21.15.31:0/40748858 learned_addr learned my addr 172.21.15.31:0/40748858 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c4102570 0x7f93c41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c3fff640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c4102ef0 0x7f93c419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c37fe640 1 -- 172.21.15.31:0/40748858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c4102570 msgr2=0x7f93c41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c37fe640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c4102570 0x7f93c41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c37fe640 1 -- 172.21.15.31:0/40748858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c4102ef0 msgr2=0x7f93c419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c37fe640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c4102ef0 0x7f93c419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c37fe640 1 -- 172.21.15.31:0/40748858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c419d420 con 0x7f93c4100e20 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.958+0000 7f93c37fe640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 0x7f93c4198470 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f93b4006010 tx=0x7f93b4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.959+0000 7f93c0ff9640 1 -- 172.21.15.31:0/40748858 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b403f070 con 0x7f93c4100e20 2024-09-16T00:41:53.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.959+0000 7f93c9f58640 1 -- 172.21.15.31:0/40748858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c419d6b0 con 0x7f93c4100e20 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.959+0000 7f93c0ff9640 1 -- 172.21.15.31:0/40748858 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b4031c30 con 0x7f93c4100e20 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.959+0000 7f93c0ff9640 1 -- 172.21.15.31:0/40748858 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4035030 con 0x7f93c4100e20 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.959+0000 7f93c9f58640 1 -- 172.21.15.31:0/40748858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c41a4ff0 con 0x7f93c4100e20 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.961+0000 7f93c0ff9640 1 -- 172.21.15.31:0/40748858 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f93b4042070 con 0x7f93c4100e20 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.961+0000 7f93cc331740 1 Processor -- start 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93cc331740 1 -- start start 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93cc331740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee78f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93cc331740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee78fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93cc331740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93cc331740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557fee6f52f0 con 0x557fee79e780 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93cc331740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557fee69ccf0 con 0x557fee790b40 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93cc331740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557fee6f12b0 con 0x557fee78f5d0 2024-09-16T00:41:53.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c0ff9640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93ac0a3240 0x7f93ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c37fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee78fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42490/0 (socket says 172.21.15.31:42490) 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c2ffd640 1 -- 172.21.15.31:0/1683526490 learned_addr learned my addr 172.21.15.31:0/1683526490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c3fff640 1 --2- 172.21.15.31:0/1683526490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee78f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c3fff640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93ac0a3240 0x7f93ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c0ff9640 1 -- 172.21.15.31:0/40748858 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f93b40f03d0 con 0x7f93c4100e20 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c37fe640 1 -- 172.21.15.31:0/1683526490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 msgr2=0x557fee78f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c37fe640 1 --2- 172.21.15.31:0/1683526490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee78f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c37fe640 1 -- 172.21.15.31:0/1683526490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 msgr2=0x557fee7a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c37fe640 1 --2- 172.21.15.31:0/1683526490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.962+0000 7f93c37fe640 1 -- 172.21.15.31:0/1683526490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fee6813a0 con 0x557fee78f5d0 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.963+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/1683526490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.963+0000 7f93c37fe640 1 --2- 172.21.15.31:0/1683526490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee78fa10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f93b40edb70 tx=0x7f93b408f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd21b6fd8c9b15f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.963+0000 7f93c3fff640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93ac0a3240 0x7f93ac0a5700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009fd0 tx=0x7f93bc009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.963+0000 7f93a9ffb640 1 -- 172.21.15.31:0/1683526490 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4049070 con 0x557fee78f5d0 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.963+0000 7f93a9ffb640 1 -- 172.21.15.31:0/1683526490 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b40bcd40 con 0x557fee78f5d0 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.963+0000 7f93a9ffb640 1 -- 172.21.15.31:0/1683526490 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b40c52b0 con 0x557fee78f5d0 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 -- 172.21.15.31:0/1683526490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 msgr2=0x557fee78fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 --2- 172.21.15.31:0/1683526490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee78fa10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f93b40edb70 tx=0x7f93b408f4c0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 -- 172.21.15.31:0/1683526490 shutdown_connections 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 --2- 172.21.15.31:0/1683526490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 --2- 172.21.15.31:0/1683526490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee78fa10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 --2- 172.21.15.31:0/1683526490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee78f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 -- 172.21.15.31:0/1683526490 >> 172.21.15.31:0/1683526490 conn(0x557fee758310 msgr2=0x557fee78bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 -- 172.21.15.31:0/1683526490 shutdown_connections 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 -- 172.21.15.31:0/1683526490 wait complete. 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 -- start start 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.964+0000 7f93cc331740 1 Processor -- start 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93cc331740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93cc331740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee79b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93cc331740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee79b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93cc331740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557fee7a1240 con 0x557fee79e780 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93cc331740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557fee6b0060 con 0x557fee790b40 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93cc331740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557fee7a13c0 con 0x557fee78f5d0 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93cc331740 10 client.0 _ll_register_callbacks cb 0x557fee4038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c37fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee79b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c37fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:53628/0 (socket says 172.21.15.31:53628) 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c37fe640 1 -- 172.21.15.31:0/3411631776 learned_addr learned my addr 172.21.15.31:0/3411631776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee79b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c37fe640 1 -- 172.21.15.31:0/3411631776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 msgr2=0x557fee79b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c37fe640 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee79b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c37fe640 1 -- 172.21.15.31:0/3411631776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 msgr2=0x557fee79b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c37fe640 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee79b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c37fe640 1 -- 172.21.15.31:0/3411631776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fee4663c0 con 0x557fee78f5d0 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c3fff640 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee79b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.965+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee79b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.966+0000 7f93c37fe640 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79e160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f93b40c4d80 tx=0x7f93b40eda90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909edead99e1868a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.966+0000 7f93a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.966+0000 7f93a2ffd640 1 -- 172.21.15.31:0/3411631776 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4049040 con 0x557fee78f5d0 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.966+0000 7f93a2ffd640 1 -- 172.21.15.31:0/3411631776 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f93b40ba940 con 0x557fee78f5d0 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.966+0000 7f93cc331740 1 -- 172.21.15.31:0/3411631776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b40bcd40 con 0x557fee78f5d0 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.966+0000 7f93a2ffd640 1 -- 172.21.15.31:0/3411631776 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f93b40bc640 con 0x557fee78f5d0 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.966+0000 7f93cc331740 10 client.25006 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.966+0000 7f93cc331740 1 -- 172.21.15.31:0/3411631776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557fee7a1240 con 0x557fee78f5d0 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.966+0000 7f93a2ffd640 1 -- 172.21.15.31:0/3411631776 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b40bcd40 con 0x557fee78f5d0 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93a2ffd640 1 -- 172.21.15.31:0/3411631776 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f93b403b4f0 con 0x557fee78f5d0 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93cc331740 10 client.25006 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93cc331740 10 client.25006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93cc331740 20 client.25006 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93cc331740 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557fee7ade40 0x557fee7b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93cc331740 4 client.25006 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93cc331740 1 -- 172.21.15.31:0/3411631776 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fee702860 con 0x557fee7ade40 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557fee7ade40 0x557fee7b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93a2ffd640 10 client.25006 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.967+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557fee7ade40 0x557fee7b0260 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f60511bfb9476 server_cookie=ad6a9b9fae78a89 in_seq=0 out_seq=0 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93a2ffd640 1 -- 172.21.15.31:0/3411631776 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fee702860 con 0x557fee7ade40 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93a2ffd640 10 client.25006 handle_command_reply: tid=0 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93cc331740 1 client.25006 shutdown 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93cc331740 1 -- 172.21.15.31:0/3411631776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 msgr2=0x557fee79e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93cc331740 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79e160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f93b40c4d80 tx=0x7f93b40eda90 comp rx=0 tx=0).stop 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93cc331740 1 -- 172.21.15.31:0/3411631776 shutdown_connections 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93cc331740 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee79b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93cc331740 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee79b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93cc331740 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557fee7ade40 0x557fee7b0260 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93cc331740 1 --2- 172.21.15.31:0/3411631776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79e160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93cc331740 1 -- 172.21.15.31:0/3411631776 >> 172.21.15.31:0/3411631776 conn(0x557fee758310 msgr2=0x557fee791960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.968+0000 7f93a2ffd640 0 client.25006 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.969+0000 7f93cc331740 1 -- 172.21.15.31:0/3411631776 shutdown_connections 2024-09-16T00:41:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.969+0000 7f93cc331740 1 -- 172.21.15.31:0/3411631776 wait complete. 2024-09-16T00:41:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.969+0000 7f93cc331740 20 client.25006 trim_cache size 0 max 16384 2024-09-16T00:41:53.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93cc331740 1 Processor -- start 2024-09-16T00:41:53.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93cc331740 1 -- start start 2024-09-16T00:41:53.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93cc331740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93cc331740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee7aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93cc331740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93cc331740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41aa980 con 0x557fee79e780 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93cc331740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41ad200 con 0x557fee790b40 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93cc331740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41aaff0 con 0x557fee78f5d0 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42514/0 (socket says 172.21.15.31:42514) 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c3fff640 1 -- 172.21.15.31:0/2155289506 learned_addr learned my addr 172.21.15.31:0/2155289506 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c37fe640 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee7aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c3fff640 1 -- 172.21.15.31:0/2155289506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 msgr2=0x557fee79b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c3fff640 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c3fff640 1 -- 172.21.15.31:0/2155289506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 msgr2=0x557fee7aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c3fff640 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee7aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c3fff640 1 -- 172.21.15.31:0/2155289506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fee78c960 con 0x557fee79e780 2024-09-16T00:41:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee7aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.985+0000 7f93c37fe640 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.986+0000 7f93c3fff640 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7b5b30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f93bc012d90 tx=0x7f93bc009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dda1b28fd0d84b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.986+0000 7f93a97fa640 1 -- 172.21.15.31:0/2155289506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc022980 con 0x557fee79e780 2024-09-16T00:41:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.986+0000 7f93a97fa640 1 -- 172.21.15.31:0/2155289506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93bc02a3e0 con 0x557fee79e780 2024-09-16T00:41:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.986+0000 7f93a97fa640 1 -- 172.21.15.31:0/2155289506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc011600 con 0x557fee79e780 2024-09-16T00:41:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 -- 172.21.15.31:0/2155289506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 msgr2=0x557fee7b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:53.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7b5b30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f93bc012d90 tx=0x7f93bc009a00 comp rx=0 tx=0).stop 2024-09-16T00:41:53.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 -- 172.21.15.31:0/2155289506 shutdown_connections 2024-09-16T00:41:53.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee7b5b30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee790b40 0x557fee7aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 --2- 172.21.15.31:0/2155289506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee78f5d0 0x557fee79b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 -- 172.21.15.31:0/2155289506 >> 172.21.15.31:0/2155289506 conn(0x557fee758310 msgr2=0x557fee79acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:53.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 -- 172.21.15.31:0/2155289506 shutdown_connections 2024-09-16T00:41:53.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 -- 172.21.15.31:0/2155289506 wait complete. 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 -- start start 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.987+0000 7f93cc331740 1 Processor -- start 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93cc331740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee78f5d0 0x557fee7a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93cc331740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee790b40 0x557fee7a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93cc331740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee816af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93cc331740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41ab6c0 con 0x557fee79e780 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93cc331740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41acb30 con 0x557fee78f5d0 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93cc331740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41abd90 con 0x557fee790b40 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93cc331740 10 client.0 _ll_register_callbacks cb 0x7f93b800a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee78f5d0 0x557fee7a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee816af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee78f5d0 0x557fee7a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:44804/0 (socket says 172.21.15.31:44804) 2024-09-16T00:41:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c37fe640 1 -- 172.21.15.31:0/669085208 learned_addr learned my addr 172.21.15.31:0/669085208 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee790b40 0x557fee7a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c37fe640 1 -- 172.21.15.31:0/669085208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee790b40 msgr2=0x557fee7a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c37fe640 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee790b40 0x557fee7a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c37fe640 1 -- 172.21.15.31:0/669085208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 msgr2=0x557fee816af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c37fe640 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee816af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c37fe640 1 -- 172.21.15.31:0/669085208 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fee7a2520 con 0x557fee78f5d0 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.988+0000 7f93c37fe640 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee78f5d0 0x557fee7a16f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93b40ede90 tx=0x7f93b4005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f991339a7164cc4a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93a2ffd640 1 -- 172.21.15.31:0/669085208 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4049040 con 0x557fee78f5d0 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93a2ffd640 1 -- 172.21.15.31:0/669085208 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f93b4031750 con 0x557fee78f5d0 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93cc331740 1 -- 172.21.15.31:0/669085208 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93bc02a3e0 con 0x557fee78f5d0 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93a2ffd640 1 -- 172.21.15.31:0/669085208 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f93b40bbfa0 con 0x557fee78f5d0 2024-09-16T00:41:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93cc331740 10 client.25067 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93cc331740 1 -- 172.21.15.31:0/669085208 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557fee818160 con 0x557fee78f5d0 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93a2ffd640 1 -- 172.21.15.31:0/669085208 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93bc02a3e0 con 0x557fee78f5d0 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93a2ffd640 1 -- 172.21.15.31:0/669085208 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f93b40bba90 con 0x557fee78f5d0 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93cc331740 10 client.25067 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93cc331740 10 client.25067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93cc331740 20 client.25067 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93cc331740 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557fee8185e0 0x557fee81a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93cc331740 4 client.25067 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.989+0000 7f93cc331740 1 -- 172.21.15.31:0/669085208 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fee6fcd70 con 0x557fee8185e0 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557fee8185e0 0x557fee81a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93c2ffd640 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557fee8185e0 0x557fee81a9d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aabc73eec9cf4ac3 server_cookie=b68139e661fe7886 in_seq=0 out_seq=0 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93a2ffd640 10 client.25067 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93a2ffd640 1 -- 172.21.15.31:0/669085208 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fee6fcd70 con 0x557fee8185e0 2024-09-16T00:41:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93a2ffd640 10 client.25067 handle_command_reply: tid=0 2024-09-16T00:41:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93cc331740 1 client.25067 shutdown 2024-09-16T00:41:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93cc331740 1 -- 172.21.15.31:0/669085208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee78f5d0 msgr2=0x557fee7a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93cc331740 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee78f5d0 0x557fee7a16f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93b40ede90 tx=0x7f93b4005e20 comp rx=0 tx=0).stop 2024-09-16T00:41:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93cc331740 1 -- 172.21.15.31:0/669085208 shutdown_connections 2024-09-16T00:41:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93cc331740 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fee79e780 0x557fee816af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93cc331740 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557fee8185e0 0x557fee81a9d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93cc331740 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557fee790b40 0x557fee7a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.990+0000 7f93a2ffd640 0 client.25067 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93cc331740 1 --2- 172.21.15.31:0/669085208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fee78f5d0 0x557fee7a16f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93cc331740 1 -- 172.21.15.31:0/669085208 >> 172.21.15.31:0/669085208 conn(0x557fee758310 msgr2=0x557fee799ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93cc331740 1 -- 172.21.15.31:0/669085208 shutdown_connections 2024-09-16T00:41:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93cc331740 1 -- 172.21.15.31:0/669085208 wait complete. 2024-09-16T00:41:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93cc331740 20 client.25067 trim_cache size 0 max 16384 2024-09-16T00:41:54.002 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:41:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93c9f58640 1 -- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93ac0a3240 msgr2=0x7f93ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93c9f58640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93ac0a3240 0x7f93ac0a5700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009fd0 tx=0x7f93bc009600 comp rx=0 tx=0).stop 2024-09-16T00:41:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93c9f58640 1 -- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 msgr2=0x7f93c4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93c9f58640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 0x7f93c4198470 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f93b4006010 tx=0x7f93b4003c10 comp rx=0 tx=0).stop 2024-09-16T00:41:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93c9f58640 1 -- 172.21.15.31:0/40748858 shutdown_connections 2024-09-16T00:41:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93c9f58640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c4102ef0 0x7f93c419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93c9f58640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93ac0a3240 0x7f93ac0a5700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93c9f58640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c4102570 0x7f93c41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.991+0000 7f93c9f58640 1 --2- 172.21.15.31:0/40748858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c4100e20 0x7f93c4198470 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.992+0000 7f93c9f58640 1 -- 172.21.15.31:0/40748858 >> 172.21.15.31:0/40748858 conn(0x7f93c40fc7d0 msgr2=0x7f93c406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.992+0000 7f93c9f58640 1 -- 172.21.15.31:0/40748858 shutdown_connections 2024-09-16T00:41:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:53.992+0000 7f93c9f58640 1 -- 172.21.15.31:0/40748858 wait complete. 2024-09-16T00:41:54.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:54 smithi031 ceph-mon[30956]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-16T00:41:55.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:54 smithi081 ceph-mon[35539]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-16T00:41:55.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:54 smithi102 ceph-mon[35364]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-16T00:41:55.817 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:41:55.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:41:55.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:55 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:41:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:56.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:55 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:41:56.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:55 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.768+0000 7f8a07905640 1 -- 172.21.15.31:0/3970866237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a00102dd0 msgr2=0x7f8a001031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.768+0000 7f8a07905640 1 --2- 172.21.15.31:0/3970866237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a00102dd0 0x7f8a001031b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009900 tx=0x7f89fc02a3e0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.768+0000 7f8a07905640 1 -- 172.21.15.31:0/3970866237 shutdown_connections 2024-09-16T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.768+0000 7f8a07905640 1 --2- 172.21.15.31:0/3970866237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a00108300 0x7f8a0010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.768+0000 7f8a07905640 1 --2- 172.21.15.31:0/3970866237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a00103780 0x7f8a00107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.768+0000 7f8a07905640 1 --2- 172.21.15.31:0/3970866237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a00102dd0 0x7f8a001031b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.768+0000 7f8a07905640 1 -- 172.21.15.31:0/3970866237 >> 172.21.15.31:0/3970866237 conn(0x7f8a000fc7d0 msgr2=0x7f8a000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.768+0000 7f8a07905640 1 -- 172.21.15.31:0/3970866237 shutdown_connections 2024-09-16T00:41:56.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.768+0000 7f8a07905640 1 -- 172.21.15.31:0/3970866237 wait complete. 2024-09-16T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.769+0000 7f8a07905640 1 Processor -- start 2024-09-16T00:41:56.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.769+0000 7f8a07905640 1 -- start start 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.770+0000 7f8a07905640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a00102dd0 0x7f8a00198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.770+0000 7f8a07905640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a00103780 0x7f8a00198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.770+0000 7f8a07905640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a00108300 0x7f8a0019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.770+0000 7f8a07905640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00112100 con 0x7f8a00102dd0 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.770+0000 7f8a07905640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00111f80 con 0x7f8a00108300 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.770+0000 7f8a07905640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00112280 con 0x7f8a00103780 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.770+0000 7f8a056a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a00102dd0 0x7f8a00198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.770+0000 7f8a056a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a00102dd0 0x7f8a00198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42554/0 (socket says 172.21.15.31:42554) 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.770+0000 7f8a056a1640 1 -- 172.21.15.31:0/814632641 learned_addr learned my addr 172.21.15.31:0/814632641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.771+0000 7f8a04ea0640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a00103780 0x7f8a00198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.771+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a00108300 0x7f8a0019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.771+0000 7f8a056a1640 1 -- 172.21.15.31:0/814632641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a00103780 msgr2=0x7f8a00198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.772+0000 7f8a056a1640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a00103780 0x7f8a00198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.772+0000 7f8a056a1640 1 -- 172.21.15.31:0/814632641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a00108300 msgr2=0x7f8a0019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.772+0000 7f8a056a1640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a00108300 0x7f8a0019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.772+0000 7f8a056a1640 1 -- 172.21.15.31:0/814632641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0019d4f0 con 0x7f8a00102dd0 2024-09-16T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.772+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a00108300 0x7f8a0019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.772+0000 7f8a04ea0640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a00103780 0x7f8a00198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.772+0000 7f8a056a1640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a00102dd0 0x7f8a00198540 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89fc02a9d0 tx=0x7f89fc003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.773+0000 7f89f67fc640 1 -- 172.21.15.31:0/814632641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc03f070 con 0x7f8a00102dd0 2024-09-16T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.773+0000 7f8a07905640 1 -- 172.21.15.31:0/814632641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0019d780 con 0x7f8a00102dd0 2024-09-16T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.773+0000 7f8a07905640 1 -- 172.21.15.31:0/814632641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a00100960 con 0x7f8a00102dd0 2024-09-16T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.773+0000 7f89f67fc640 1 -- 172.21.15.31:0/814632641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89fc03b940 con 0x7f8a00102dd0 2024-09-16T00:41:56.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.773+0000 7f89f67fc640 1 -- 172.21.15.31:0/814632641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc049060 con 0x7f8a00102dd0 2024-09-16T00:41:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.776+0000 7f8a09cde740 1 Processor -- start 2024-09-16T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.776+0000 7f8a09cde740 1 -- start start 2024-09-16T00:41:56.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faedf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 0x55c70faf8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faf8d40 0x55c70fafb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a09cde740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c70fa53280 con 0x55c70faee480 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a09cde740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c70f9fab10 con 0x55c70faf8d40 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a09cde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c70fa4f240 con 0x55c70faedb60 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f89f67fc640 1 -- 172.21.15.31:0/814632641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f89fc042070 con 0x7f8a00102dd0 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a05ea2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faedf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a05ea2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faedf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:53674/0 (socket says 172.21.15.31:53674) 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a05ea2640 1 -- 172.21.15.31:0/2522051296 learned_addr learned my addr 172.21.15.31:0/2522051296 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a04ea0640 1 --2- 172.21.15.31:0/2522051296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faf8d40 0x55c70fafb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a05ea2640 1 -- 172.21.15.31:0/2522051296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faf8d40 msgr2=0x55c70fafb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/2522051296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faf8d40 0x55c70fafb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a05ea2640 1 -- 172.21.15.31:0/2522051296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 msgr2=0x55c70faf8800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/2522051296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 0x55c70faf8800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.777+0000 7f8a05ea2640 1 -- 172.21.15.31:0/2522051296 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70fa071f0 con 0x55c70faedb60 2024-09-16T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.778+0000 7f89f67fc640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89ec0a32c0 0x7f89ec0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.778+0000 7f8a056a1640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89ec0a32c0 0x7f89ec0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.778+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/2522051296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faedf40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009f30 tx=0x7f89f8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9042cfdde3b10fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.778+0000 7f89eb7fe640 1 -- 172.21.15.31:0/2522051296 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8016070 con 0x55c70faedb60 2024-09-16T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.778+0000 7f89eb7fe640 1 -- 172.21.15.31:0/2522051296 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f801bb40 con 0x55c70faedb60 2024-09-16T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.778+0000 7f89eb7fe640 1 -- 172.21.15.31:0/2522051296 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8012cb0 con 0x55c70faedb60 2024-09-16T00:41:56.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.778+0000 7f89f67fc640 1 -- 172.21.15.31:0/814632641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f89fc0f0380 con 0x7f8a00102dd0 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.779+0000 7f8a09cde740 1 -- 172.21.15.31:0/2522051296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 msgr2=0x55c70faedf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.779+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2522051296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faedf40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009f30 tx=0x7f89f8010800 comp rx=0 tx=0).stop 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.779+0000 7f8a056a1640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89ec0a32c0 0x7f89ec0a5780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0b82d0 tx=0x7f89fc0b81e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.779+0000 7f8a09cde740 1 -- 172.21.15.31:0/2522051296 shutdown_connections 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.779+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2522051296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faf8d40 0x55c70fafb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.779+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2522051296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 0x55c70faf8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.779+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2522051296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faedf40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.779+0000 7f8a09cde740 1 -- 172.21.15.31:0/2522051296 >> 172.21.15.31:0/2522051296 conn(0x55c70faea0f0 msgr2=0x55c70faea4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.780+0000 7f8a09cde740 1 -- 172.21.15.31:0/2522051296 shutdown_connections 2024-09-16T00:41:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.780+0000 7f8a09cde740 1 -- 172.21.15.31:0/2522051296 wait complete. 2024-09-16T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.780+0000 7f8a09cde740 1 -- start start 2024-09-16T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.780+0000 7f8a09cde740 1 Processor -- start 2024-09-16T00:41:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 0x55c70faf6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faf8d40 0x55c70faf6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a09cde740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c70fb04120 con 0x55c70faee480 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a09cde740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c70fa0df10 con 0x55c70faf8d40 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a09cde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c70fb042a0 con 0x55c70faedb60 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a09cde740 10 client.0 _ll_register_callbacks cb 0x55c70f9d6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a04ea0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 0x55c70faf6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a05ea2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a04ea0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 0x55c70faf6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42564/0 (socket says 172.21.15.31:42564) 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.781+0000 7f8a04ea0640 1 -- 172.21.15.31:0/1133885556 learned_addr learned my addr 172.21.15.31:0/1133885556 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f8a04ea0640 1 -- 172.21.15.31:0/1133885556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 msgr2=0x55c70faf5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f8a04ea0640 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f8a04ea0640 1 -- 172.21.15.31:0/1133885556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faf8d40 msgr2=0x55c70faf6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f8a04ea0640 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faf8d40 0x55c70faf6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f8a04ea0640 1 -- 172.21.15.31:0/1133885556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70f7c46a0 con 0x55c70faee480 2024-09-16T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f8a04ea0640 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 0x55c70faf6210 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f89f0010f40 tx=0x7f89f000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c43e4d749bc8691 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f89e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f89e8ff9640 1 -- 172.21.15.31:0/1133885556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0005020 con 0x55c70faee480 2024-09-16T00:41:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f89e8ff9640 1 -- 172.21.15.31:0/1133885556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f89f00051c0 con 0x55c70faee480 2024-09-16T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.782+0000 7f8a09cde740 1 -- 172.21.15.31:0/1133885556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f801bb40 con 0x55c70faee480 2024-09-16T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.783+0000 7f89e8ff9640 1 -- 172.21.15.31:0/1133885556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f89f00053c0 con 0x55c70faee480 2024-09-16T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.783+0000 7f8a09cde740 10 client.15369 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.783+0000 7f8a09cde740 1 -- 172.21.15.31:0/1133885556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c70fafb700 con 0x55c70faee480 2024-09-16T00:41:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.783+0000 7f89e8ff9640 1 -- 172.21.15.31:0/1133885556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f801bb40 con 0x55c70faee480 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.783+0000 7f89e8ff9640 1 -- 172.21.15.31:0/1133885556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f89f002a690 con 0x55c70faee480 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.783+0000 7f8a09cde740 10 client.15369 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.783+0000 7f8a09cde740 10 client.15369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.783+0000 7f8a09cde740 20 client.15369 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.784+0000 7f8a09cde740 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c70fafcc60 0x55c70faff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.784+0000 7f8a09cde740 4 client.15369 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.784+0000 7f8a09cde740 1 -- 172.21.15.31:0/1133885556 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c70fa607f0 con 0x55c70fafcc60 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.784+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c70fafcc60 0x55c70faff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.784+0000 7f89e8ff9640 10 client.15369 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.784+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c70fafcc60 0x55c70faff080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217c65a455fa7d0 server_cookie=26fbf46a32e78fed in_seq=0 out_seq=0 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f89e8ff9640 1 -- 172.21.15.31:0/1133885556 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c70fa607f0 con 0x55c70fafcc60 2024-09-16T00:41:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f89e8ff9640 10 client.15369 handle_command_reply: tid=0 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 client.15369 shutdown 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 -- 172.21.15.31:0/1133885556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 msgr2=0x55c70faf6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 0x55c70faf6210 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f89f0010f40 tx=0x7f89f000f7f0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 -- 172.21.15.31:0/1133885556 shutdown_connections 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faf8d40 0x55c70faf6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faee480 0x55c70faf6210 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c70fafcc60 0x55c70faff080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 --2- 172.21.15.31:0/1133885556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f89e8ff9640 0 client.15369 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 -- 172.21.15.31:0/1133885556 >> 172.21.15.31:0/1133885556 conn(0x55c70faea0f0 msgr2=0x55c70faefe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 -- 172.21.15.31:0/1133885556 shutdown_connections 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.785+0000 7f8a09cde740 1 -- 172.21.15.31:0/1133885556 wait complete. 2024-09-16T00:41:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.786+0000 7f8a09cde740 20 client.15369 trim_cache size 0 max 16384 2024-09-16T00:41:56.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.811+0000 7f8a09cde740 1 Processor -- start 2024-09-16T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.811+0000 7f8a09cde740 1 -- start start 2024-09-16T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.811+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 0x55c70faf65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 0x55c70fb14dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a09cde740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89e400dc50 con 0x55c70faf8d40 2024-09-16T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a09cde740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f89e40104d0 con 0x55c70faee480 2024-09-16T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a09cde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f89e400e2c0 con 0x55c70faedb60 2024-09-16T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:53692/0 (socket says 172.21.15.31:53692) 2024-09-16T00:41:56.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 -- 172.21.15.31:0/3402552847 learned_addr learned my addr 172.21.15.31:0/3402552847 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a04ea0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 0x55c70faf65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/3402552847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 0x55c70fb14dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 -- 172.21.15.31:0/3402552847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 msgr2=0x55c70faf65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/3402552847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 0x55c70faf65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 -- 172.21.15.31:0/3402552847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 msgr2=0x55c70fb14dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/3402552847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 0x55c70fb14dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 -- 172.21.15.31:0/3402552847 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70faeb1a0 con 0x55c70faedb60 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.812+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/3402552847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf05c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009f30 tx=0x7f89f801d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19f6fa5be381cee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.813+0000 7f89eaffd640 1 -- 172.21.15.31:0/3402552847 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8016050 con 0x55c70faedb60 2024-09-16T00:41:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.813+0000 7f89eaffd640 1 -- 172.21.15.31:0/3402552847 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f8012e20 con 0x55c70faedb60 2024-09-16T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.813+0000 7f89eaffd640 1 -- 172.21.15.31:0/3402552847 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8028a40 con 0x55c70faedb60 2024-09-16T00:41:56.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.814+0000 7f8a09cde740 1 -- 172.21.15.31:0/3402552847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 msgr2=0x55c70faf05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.814+0000 7f8a09cde740 1 --2- 172.21.15.31:0/3402552847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf05c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009f30 tx=0x7f89f801d040 comp rx=0 tx=0).stop 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.814+0000 7f8a09cde740 1 -- 172.21.15.31:0/3402552847 shutdown_connections 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.814+0000 7f8a09cde740 1 --2- 172.21.15.31:0/3402552847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 0x55c70fb14dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.814+0000 7f8a09cde740 1 --2- 172.21.15.31:0/3402552847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 0x55c70faf65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.814+0000 7f8a09cde740 1 --2- 172.21.15.31:0/3402552847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70faf05c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.814+0000 7f8a09cde740 1 -- 172.21.15.31:0/3402552847 >> 172.21.15.31:0/3402552847 conn(0x55c70faea0f0 msgr2=0x55c70faf7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.814+0000 7f8a09cde740 1 -- 172.21.15.31:0/3402552847 shutdown_connections 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.814+0000 7f8a09cde740 1 -- 172.21.15.31:0/3402552847 wait complete. 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.815+0000 7f8a09cde740 1 -- start start 2024-09-16T00:41:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.815+0000 7f8a09cde740 1 Processor -- start 2024-09-16T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.815+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70fb12b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.815+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 0x55c70fb130a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.815+0000 7f8a09cde740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 0x55c70fb135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.815+0000 7f8a09cde740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89e400e990 con 0x55c70faf8d40 2024-09-16T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.815+0000 7f8a09cde740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f89e400fe00 con 0x55c70faee480 2024-09-16T00:41:56.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.815+0000 7f8a09cde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f89e400f060 con 0x55c70faedb60 2024-09-16T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a09cde740 10 client.0 _ll_register_callbacks cb 0x7f89f000abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a04ea0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 0x55c70fb130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a04ea0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 0x55c70fb130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:44834/0 (socket says 172.21.15.31:44834) 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a04ea0640 1 -- 172.21.15.31:0/2573157912 learned_addr learned my addr 172.21.15.31:0/2573157912 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 0x55c70fb135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70fb12b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a05ea2640 1 -- 172.21.15.31:0/2573157912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 msgr2=0x55c70fb12b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70fb12b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a05ea2640 1 -- 172.21.15.31:0/2573157912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 msgr2=0x55c70fb130a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 0x55c70fb130a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a05ea2640 1 -- 172.21.15.31:0/2573157912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70fb72e50 con 0x55c70faf8d40 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f8a05ea2640 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 0x55c70fb135e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f89f8010d80 tx=0x7f89f8028620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d9541870de9783 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.816+0000 7f89e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.817+0000 7f89e8ff9640 1 -- 172.21.15.31:0/2573157912 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8016070 con 0x55c70faf8d40 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.817+0000 7f8a09cde740 1 -- 172.21.15.31:0/2573157912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f8012e20 con 0x55c70faf8d40 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.817+0000 7f89e8ff9640 1 -- 172.21.15.31:0/2573157912 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f89f8012940 con 0x55c70faf8d40 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.817+0000 7f89e8ff9640 1 -- 172.21.15.31:0/2573157912 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f89f803c040 con 0x55c70faf8d40 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.817+0000 7f8a09cde740 10 client.15375 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.817+0000 7f8a09cde740 1 -- 172.21.15.31:0/2573157912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c70fb740f0 con 0x55c70faf8d40 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.817+0000 7f89e8ff9640 1 -- 172.21.15.31:0/2573157912 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f8012e20 con 0x55c70faf8d40 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.818+0000 7f89e8ff9640 1 -- 172.21.15.31:0/2573157912 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f89f8031070 con 0x55c70faf8d40 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.818+0000 7f8a09cde740 10 client.15375 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.818+0000 7f8a09cde740 10 client.15375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.818+0000 7f8a09cde740 20 client.15375 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.818+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c70fb74570 0x55c70fb76990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.818+0000 7f8a09cde740 4 client.15375 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.818+0000 7f8a09cde740 1 -- 172.21.15.31:0/2573157912 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c70fa5ad00 con 0x55c70fb74570 2024-09-16T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.818+0000 7f8a04ea0640 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c70fb74570 0x55c70fb76990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.819+0000 7f8a04ea0640 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c70fb74570 0x55c70fb76990 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c15d974ac99d469 server_cookie=49bbfadd01b6ed1d in_seq=0 out_seq=0 2024-09-16T00:41:56.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.819+0000 7f89e8ff9640 10 client.15375 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:56.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.820+0000 7f89e8ff9640 1 -- 172.21.15.31:0/2573157912 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c70fa5ad00 con 0x55c70fb74570 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.820+0000 7f89e8ff9640 10 client.15375 handle_command_reply: tid=0 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.820+0000 7f8a09cde740 1 client.15375 shutdown 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.820+0000 7f8a09cde740 1 -- 172.21.15.31:0/2573157912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 msgr2=0x55c70fb135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.820+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 0x55c70fb135e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f89f8010d80 tx=0x7f89f8028620 comp rx=0 tx=0).stop 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.821+0000 7f8a09cde740 1 -- 172.21.15.31:0/2573157912 shutdown_connections 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.821+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c70faf8d40 0x55c70fb135e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.821+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70faee480 0x55c70fb130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.821+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c70fb74570 0x55c70fb76990 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.821+0000 7f8a09cde740 1 --2- 172.21.15.31:0/2573157912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c70faedb60 0x55c70fb12b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.821+0000 7f89e8ff9640 0 client.15375 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.821+0000 7f8a09cde740 1 -- 172.21.15.31:0/2573157912 >> 172.21.15.31:0/2573157912 conn(0x55c70faea0f0 msgr2=0x55c70fae8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.822+0000 7f8a09cde740 1 -- 172.21.15.31:0/2573157912 shutdown_connections 2024-09-16T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.822+0000 7f8a09cde740 1 -- 172.21.15.31:0/2573157912 wait complete. 2024-09-16T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.822+0000 7f8a09cde740 20 client.15375 trim_cache size 0 max 16384 2024-09-16T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "71f6f529-f6e0-4d86-8923-8e38ad3a786e", 2024-09-16T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:41:56.826 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 -- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89ec0a32c0 msgr2=0x7f89ec0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89ec0a32c0 0x7f89ec0a5780 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0b82d0 tx=0x7f89fc0b81e0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 -- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a00102dd0 msgr2=0x7f8a00198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a00102dd0 0x7f8a00198540 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89fc02a9d0 tx=0x7f89fc003b60 comp rx=0 tx=0).stop 2024-09-16T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 -- 172.21.15.31:0/814632641 shutdown_connections 2024-09-16T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a00108300 0x7f8a0019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89ec0a32c0 0x7f89ec0a5780 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a00103780 0x7f8a00198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 --2- 172.21.15.31:0/814632641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a00102dd0 0x7f8a00198540 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 -- 172.21.15.31:0/814632641 >> 172.21.15.31:0/814632641 conn(0x7f8a000fc7d0 msgr2=0x7f8a000fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 -- 172.21.15.31:0/814632641 shutdown_connections 2024-09-16T00:41:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:56.824+0000 7f8a07905640 1 -- 172.21.15.31:0/814632641 wait complete. 2024-09-16T00:41:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:41:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:41:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:41:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:41:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:41:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:41:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:57.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:41:57.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:57 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:41:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:41:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:41:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:41:57.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:41:57.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:41:57.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:57.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:41:57.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:57 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:57.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:41:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:41:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:41:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:41:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:41:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:41:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:41:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:57 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:41:57.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:41:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:41:58.496 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:58 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:41:58.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:58 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:41:58.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:58 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:41:58.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.593+0000 7f053e4a5640 1 -- 172.21.15.31:0/1340918115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100aa0 msgr2=0x7f053810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:58.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.593+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1340918115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100aa0 0x7f053810ea20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0530009ca0 tx=0x7f053002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.593+0000 7f053e4a5640 1 -- 172.21.15.31:0/1340918115 shutdown_connections 2024-09-16T00:41:58.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.593+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1340918115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100aa0 0x7f053810ea20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.593+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1340918115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0538100100 0x7f0538100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.593+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1340918115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0538106b80 0x7f0538106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.593+0000 7f053e4a5640 1 -- 172.21.15.31:0/1340918115 >> 172.21.15.31:0/1340918115 conn(0x7f0538077230 msgr2=0x7f05380fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.593+0000 7f053e4a5640 1 -- 172.21.15.31:0/1340918115 shutdown_connections 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.593+0000 7f053e4a5640 1 -- 172.21.15.31:0/1340918115 wait complete. 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f053e4a5640 1 Processor -- start 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f053e4a5640 1 -- start start 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f053e4a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100100 0x7f05381012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f053e4a5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0538100aa0 0x7f0538101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f053e4a5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0538106b80 0x7f0538101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f053e4a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0538112050 con 0x7f0538100100 2024-09-16T00:41:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f053e4a5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0538111ed0 con 0x7f0538106b80 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f053e4a5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05381121d0 con 0x7f0538100aa0 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f0537fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100100 0x7f05381012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f0537fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100100 0x7f05381012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54960/0 (socket says 172.21.15.31:54960) 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.594+0000 7f0537fff640 1 -- 172.21.15.31:0/1167597612 learned_addr learned my addr 172.21.15.31:0/1167597612 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f05377fe640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0538100aa0 0x7f0538101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f053ca42640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0538106b80 0x7f0538101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f0537fff640 1 -- 172.21.15.31:0/1167597612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0538100aa0 msgr2=0x7f0538101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f0537fff640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0538100aa0 0x7f0538101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f0537fff640 1 -- 172.21.15.31:0/1167597612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0538106b80 msgr2=0x7f0538101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f0537fff640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0538106b80 0x7f0538101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f0537fff640 1 -- 172.21.15.31:0/1167597612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05381a6950 con 0x7f0538100100 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f053ca42640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0538106b80 0x7f0538101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f0537fff640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100100 0x7f05381012e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f052800d950 tx=0x7f052800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f05357fa640 1 -- 172.21.15.31:0/1167597612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528017030 con 0x7f0538100100 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f053e4a5640 1 -- 172.21.15.31:0/1167597612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05381a6c40 con 0x7f0538100100 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f05357fa640 1 -- 172.21.15.31:0/1167597612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0528004020 con 0x7f0538100100 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f05357fa640 1 -- 172.21.15.31:0/1167597612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05280047a0 con 0x7f0538100100 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.595+0000 7f053e4a5640 1 -- 172.21.15.31:0/1167597612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05381a7180 con 0x7f0538100100 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.598+0000 7f05357fa640 1 -- 172.21.15.31:0/1167597612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f05280041d0 con 0x7f0538100100 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.599+0000 7f054087e740 1 Processor -- start 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.599+0000 7f05357fa640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05200a32c0 0x7f05200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.599+0000 7f05357fa640 1 -- 172.21.15.31:0/1167597612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f05280cf580 con 0x7f0538100100 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.600+0000 7f054087e740 1 -- start start 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.600+0000 7f054087e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f750880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.600+0000 7f054087e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 0x55927f75b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.600+0000 7f054087e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f75b680 0x55927f75da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.600+0000 7f054087e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55927f6b62e0 con 0x55927f750dc0 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.600+0000 7f054087e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55927f65dce0 con 0x55927f7504a0 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.600+0000 7f054087e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55927f6b22a0 con 0x55927f75b680 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.600+0000 7f053ca42640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05200a32c0 0x7f05200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f0537fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f750880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f0537fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f750880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:58840/0 (socket says 172.21.15.31:58840) 2024-09-16T00:41:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f0537fff640 1 -- 172.21.15.31:0/911280163 learned_addr learned my addr 172.21.15.31:0/911280163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f053ca42640 1 --2- 172.21.15.31:0/911280163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f75b680 0x55927f75da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f0537fff640 1 -- 172.21.15.31:0/911280163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f75b680 msgr2=0x55927f75da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f0537fff640 1 --2- 172.21.15.31:0/911280163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f75b680 0x55927f75da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f0537fff640 1 -- 172.21.15.31:0/911280163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 msgr2=0x55927f75b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f0537fff640 1 --2- 172.21.15.31:0/911280163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 0x55927f75b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f0537fff640 1 -- 172.21.15.31:0/911280163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55927f642440 con 0x55927f7504a0 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f053ca42640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05200a32c0 0x7f05200a5780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0530006010 tx=0x7f0530031ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.601+0000 7f0537fff640 1 --2- 172.21.15.31:0/911280163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f750880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0528097c50 tx=0x7f052809b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed2841ffa76c97c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.602+0000 7f051e7fc640 1 -- 172.21.15.31:0/911280163 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052809bef0 con 0x55927f7504a0 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.602+0000 7f051e7fc640 1 -- 172.21.15.31:0/911280163 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052809b7f0 con 0x55927f7504a0 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.602+0000 7f051e7fc640 1 -- 172.21.15.31:0/911280163 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528084b40 con 0x55927f7504a0 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.603+0000 7f054087e740 1 -- 172.21.15.31:0/911280163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 msgr2=0x55927f750880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.603+0000 7f054087e740 1 --2- 172.21.15.31:0/911280163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f750880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0528097c50 tx=0x7f052809b5c0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.603+0000 7f054087e740 1 -- 172.21.15.31:0/911280163 shutdown_connections 2024-09-16T00:41:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.603+0000 7f054087e740 1 --2- 172.21.15.31:0/911280163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f75b680 0x55927f75da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.603+0000 7f054087e740 1 --2- 172.21.15.31:0/911280163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 0x55927f75b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.603+0000 7f054087e740 1 --2- 172.21.15.31:0/911280163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f750880 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.603+0000 7f054087e740 1 -- 172.21.15.31:0/911280163 >> 172.21.15.31:0/911280163 conn(0x55927f74ca30 msgr2=0x55927f74ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.603+0000 7f054087e740 1 -- 172.21.15.31:0/911280163 shutdown_connections 2024-09-16T00:41:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.603+0000 7f054087e740 1 -- 172.21.15.31:0/911280163 wait complete. 2024-09-16T00:41:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.604+0000 7f054087e740 1 -- start start 2024-09-16T00:41:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.604+0000 7f054087e740 1 Processor -- start 2024-09-16T00:41:58.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.604+0000 7f054087e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f75b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.604+0000 7f054087e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f750dc0 0x55927f7582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.604+0000 7f054087e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 0x55927f758880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.604+0000 7f054087e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55927f75e250 con 0x55927f75b680 2024-09-16T00:41:58.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.604+0000 7f054087e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55927f671050 con 0x55927f750dc0 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.604+0000 7f054087e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55927f75e3d0 con 0x55927f7504a0 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.604+0000 7f054087e740 10 client.0 _ll_register_callbacks cb 0x55927f6b1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05377fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f750dc0 0x55927f7582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05377fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f750dc0 0x55927f7582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:58846/0 (socket says 172.21.15.31:58846) 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05377fe640 1 -- 172.21.15.31:0/1802764348 learned_addr learned my addr 172.21.15.31:0/1802764348 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f053ca42640 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 0x55927f758880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f0537fff640 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f75b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05377fe640 1 -- 172.21.15.31:0/1802764348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 msgr2=0x55927f75b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05377fe640 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f75b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05377fe640 1 -- 172.21.15.31:0/1802764348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 msgr2=0x55927f758880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05377fe640 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 0x55927f758880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05377fe640 1 -- 172.21.15.31:0/1802764348 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55927f4273c0 con 0x55927f750dc0 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f0537fff640 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f75b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05377fe640 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f750dc0 0x55927f7582e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f052c00b8f0 tx=0x7f052c00bdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1873fc6c6beabf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05177fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05177fe640 1 -- 172.21.15.31:0/1802764348 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c005020 con 0x55927f750dc0 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.605+0000 7f05177fe640 1 -- 172.21.15.31:0/1802764348 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f052c0051c0 con 0x55927f750dc0 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.606+0000 7f054087e740 1 -- 172.21.15.31:0/1802764348 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f052809b7f0 con 0x55927f750dc0 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.606+0000 7f05177fe640 1 -- 172.21.15.31:0/1802764348 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f052c022050 con 0x55927f750dc0 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.606+0000 7f054087e740 10 client.25094 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.606+0000 7f054087e740 1 -- 172.21.15.31:0/1802764348 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55927f7724d0 con 0x55927f750dc0 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.606+0000 7f05177fe640 1 -- 172.21.15.31:0/1802764348 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052809b7f0 con 0x55927f750dc0 2024-09-16T00:41:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.606+0000 7f05177fe640 1 -- 172.21.15.31:0/1802764348 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f052c0115d0 con 0x55927f750dc0 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.607+0000 7f054087e740 10 client.25094 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.607+0000 7f054087e740 10 client.25094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.607+0000 7f054087e740 20 client.25094 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.607+0000 7f054087e740 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55927f773990 0x55927f775df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.607+0000 7f054087e740 4 client.25094 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.607+0000 7f054087e740 1 -- 172.21.15.31:0/1802764348 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55927f6c3850 con 0x55927f773990 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.607+0000 7f0537fff640 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55927f773990 0x55927f775df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.607+0000 7f0537fff640 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55927f773990 0x55927f775df0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1840d38faf003228 server_cookie=fa05457978171437 in_seq=0 out_seq=0 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.607+0000 7f05177fe640 10 client.25094 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.608+0000 7f05177fe640 1 -- 172.21.15.31:0/1802764348 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55927f6c3850 con 0x55927f773990 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.608+0000 7f05177fe640 10 client.25094 handle_command_reply: tid=0 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.608+0000 7f054087e740 1 client.25094 shutdown 2024-09-16T00:41:58.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.608+0000 7f054087e740 1 -- 172.21.15.31:0/1802764348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f750dc0 msgr2=0x55927f7582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.608+0000 7f054087e740 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f750dc0 0x55927f7582e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f052c00b8f0 tx=0x7f052c00bdb0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.608+0000 7f054087e740 1 -- 172.21.15.31:0/1802764348 shutdown_connections 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.609+0000 7f054087e740 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 0x55927f758880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.609+0000 7f054087e740 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f750dc0 0x55927f7582e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.609+0000 7f054087e740 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55927f773990 0x55927f775df0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.609+0000 7f05177fe640 0 client.25094 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.609+0000 7f054087e740 1 --2- 172.21.15.31:0/1802764348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f75b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.609+0000 7f054087e740 1 -- 172.21.15.31:0/1802764348 >> 172.21.15.31:0/1802764348 conn(0x55927f74ca30 msgr2=0x55927f752760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.609+0000 7f054087e740 1 -- 172.21.15.31:0/1802764348 shutdown_connections 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.609+0000 7f054087e740 1 -- 172.21.15.31:0/1802764348 wait complete. 2024-09-16T00:41:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.609+0000 7f054087e740 20 client.25094 trim_cache size 0 max 16384 2024-09-16T00:41:58.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.643+0000 7f054087e740 1 Processor -- start 2024-09-16T00:41:58.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.643+0000 7f054087e740 1 -- start start 2024-09-16T00:41:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.643+0000 7f054087e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f76fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.643+0000 7f054087e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 0x55927f756bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.643+0000 7f054087e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f75b680 0x55927f77b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f054087e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05381aaba0 con 0x55927f750dc0 2024-09-16T00:41:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f054087e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f05381ad400 con 0x55927f75b680 2024-09-16T00:41:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f054087e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05381ab1f0 con 0x55927f7504a0 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f05377fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 0x55927f756bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f05377fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 0x55927f756bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54988/0 (socket says 172.21.15.31:54988) 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f05377fe640 1 -- 172.21.15.31:0/3700161037 learned_addr learned my addr 172.21.15.31:0/3700161037 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f0537fff640 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f76fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f053ca42640 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f75b680 0x55927f77b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f05377fe640 1 -- 172.21.15.31:0/3700161037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 msgr2=0x55927f76fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f05377fe640 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f76fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f05377fe640 1 -- 172.21.15.31:0/3700161037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f75b680 msgr2=0x55927f77b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f05377fe640 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f75b680 0x55927f77b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f05377fe640 1 -- 172.21.15.31:0/3700161037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55927f757290 con 0x55927f750dc0 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f0537fff640 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f76fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f053ca42640 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f75b680 0x55927f77b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:41:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.644+0000 7f05377fe640 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 0x55927f756bb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f052c000c00 tx=0x7f052c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa65dc6d617614b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.645+0000 7f051dffb640 1 -- 172.21.15.31:0/3700161037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c003d60 con 0x55927f750dc0 2024-09-16T00:41:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.645+0000 7f051dffb640 1 -- 172.21.15.31:0/3700161037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052c02a030 con 0x55927f750dc0 2024-09-16T00:41:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.645+0000 7f051dffb640 1 -- 172.21.15.31:0/3700161037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c005530 con 0x55927f750dc0 2024-09-16T00:41:58.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 -- 172.21.15.31:0/3700161037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 msgr2=0x55927f756bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 0x55927f756bb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f052c000c00 tx=0x7f052c00b390 comp rx=0 tx=0).stop 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 -- 172.21.15.31:0/3700161037 shutdown_connections 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f75b680 0x55927f77b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f750dc0 0x55927f756bb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 --2- 172.21.15.31:0/3700161037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f7504a0 0x55927f76fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 -- 172.21.15.31:0/3700161037 >> 172.21.15.31:0/3700161037 conn(0x55927f74ca30 msgr2=0x55927f753900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 -- 172.21.15.31:0/3700161037 shutdown_connections 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 -- 172.21.15.31:0/3700161037 wait complete. 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.646+0000 7f054087e740 1 -- start start 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f054087e740 1 Processor -- start 2024-09-16T00:41:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f054087e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f767de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f054087e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f750dc0 0x55927f768320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f054087e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 0x55927f768860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f054087e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05381ab8c0 con 0x55927f75b680 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f054087e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f05381acd30 con 0x55927f7504a0 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f054087e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05381abf90 con 0x55927f750dc0 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f05377fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f750dc0 0x55927f768320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f054087e740 10 client.0 _ll_register_callbacks cb 0x7f052c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f05377fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f750dc0 0x55927f768320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:35768/0 (socket says 172.21.15.31:35768) 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f05377fe640 1 -- 172.21.15.31:0/529689754 learned_addr learned my addr 172.21.15.31:0/529689754 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f053ca42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 0x55927f768860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.647+0000 7f0537fff640 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f767de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f0537fff640 1 -- 172.21.15.31:0/529689754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f750dc0 msgr2=0x55927f768320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f0537fff640 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f750dc0 0x55927f768320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f0537fff640 1 -- 172.21.15.31:0/529689754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 msgr2=0x55927f768860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f0537fff640 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 0x55927f768860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f0537fff640 1 -- 172.21.15.31:0/529689754 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55927f750bf0 con 0x55927f7504a0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f0537fff640 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f767de0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0528005820 tx=0x7f0528097a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=445382d4fc181d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f05177fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f05177fe640 1 -- 172.21.15.31:0/529689754 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052809bef0 con 0x55927f7504a0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f05177fe640 1 -- 172.21.15.31:0/529689754 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f0528097e60 con 0x55927f7504a0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f054087e740 1 -- 172.21.15.31:0/529689754 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f052c02a030 con 0x55927f7504a0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f05177fe640 1 -- 172.21.15.31:0/529689754 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f05280a4460 con 0x55927f7504a0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f054087e740 10 client.25100 fetch_fsmap learned FSMap version 20 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f054087e740 1 -- 172.21.15.31:0/529689754 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55927f758bf0 con 0x55927f7504a0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.648+0000 7f05177fe640 1 -- 172.21.15.31:0/529689754 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052809b7f0 con 0x55927f7504a0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f05177fe640 1 -- 172.21.15.31:0/529689754 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f052809ad60 con 0x55927f7504a0 2024-09-16T00:41:58.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f054087e740 10 client.25100 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f054087e740 10 client.25100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f054087e740 20 client.25100 populate_metadata read hostname 'smithi031' 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f054087e740 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55927f773990 0x55927f7e0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f054087e740 4 client.25100 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f054087e740 1 -- 172.21.15.31:0/529689754 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f052c02a030 con 0x55927f773990 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f05377fe640 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55927f773990 0x55927f7e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f05177fe640 10 client.25100 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.649+0000 7f05377fe640 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55927f773990 0x55927f7e0940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708119671552c86b server_cookie=89ddfcdbf292048e in_seq=0 out_seq=0 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f05177fe640 1 -- 172.21.15.31:0/529689754 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f052c02a030 con 0x55927f773990 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f05177fe640 10 client.25100 handle_command_reply: tid=0 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 client.25100 shutdown 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 -- 172.21.15.31:0/529689754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 msgr2=0x55927f767de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:58.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f767de0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0528005820 tx=0x7f0528097a90 comp rx=0 tx=0).stop 2024-09-16T00:41:58.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 -- 172.21.15.31:0/529689754 shutdown_connections 2024-09-16T00:41:58.658 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:41:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:41:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:41:58.658 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:41:58.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55927f75b680 0x55927f768860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55927f773990 0x55927f7e0940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55927f750dc0 0x55927f768320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 --2- 172.21.15.31:0/529689754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55927f7504a0 0x55927f767de0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 -- 172.21.15.31:0/529689754 >> 172.21.15.31:0/529689754 conn(0x55927f74ca30 msgr2=0x55927f75a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:58.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f05177fe640 0 client.25100 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 -- 172.21.15.31:0/529689754 shutdown_connections 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 1 -- 172.21.15.31:0/529689754 wait complete. 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.650+0000 7f054087e740 20 client.25100 trim_cache size 0 max 16384 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 -- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05200a32c0 msgr2=0x7f05200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05200a32c0 0x7f05200a5780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0530006010 tx=0x7f0530031ce0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 -- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100100 msgr2=0x7f05381012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100100 0x7f05381012e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f052800d950 tx=0x7f052800de10 comp rx=0 tx=0).stop 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 -- 172.21.15.31:0/1167597612 shutdown_connections 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0538106b80 0x7f0538101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05200a32c0 0x7f05200a5780 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0538100aa0 0x7f0538101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 --2- 172.21.15.31:0/1167597612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0538100100 0x7f05381012e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 -- 172.21.15.31:0/1167597612 >> 172.21.15.31:0/1167597612 conn(0x7f0538077230 msgr2=0x7f0538104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:58.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.651+0000 7f053e4a5640 1 -- 172.21.15.31:0/1167597612 shutdown_connections 2024-09-16T00:41:58.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:41:58.652+0000 7f053e4a5640 1 -- 172.21.15.31:0/1167597612 wait complete. 2024-09-16T00:41:59.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:41:59 smithi031 ceph-mon[30956]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:59.470 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:41:59.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:41:59.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:41:59 smithi081 ceph-mon[35539]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:59.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:41:59 smithi102 ceph-mon[35364]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:41:59.752 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 -- 172.21.15.31:0/1928666774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20100e20 msgr2=0x7fdf20101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1928666774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20100e20 0x7fdf20101200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009900 tx=0x7fdf1002a480 comp rx=0 tx=0).stop 2024-09-16T00:42:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 -- 172.21.15.31:0/1928666774 shutdown_connections 2024-09-16T00:42:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1928666774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf20102ef0 0x7fdf2010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1928666774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20102570 0x7fdf201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1928666774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20100e20 0x7fdf20101200 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 -- 172.21.15.31:0/1928666774 >> 172.21.15.31:0/1928666774 conn(0x7fdf200fc7d0 msgr2=0x7fdf200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 -- 172.21.15.31:0/1928666774 shutdown_connections 2024-09-16T00:42:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 -- 172.21.15.31:0/1928666774 wait complete. 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.305+0000 7fdf254e9640 1 Processor -- start 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf254e9640 1 -- start start 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf254e9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20100e20 0x7fdf2010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf254e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20102570 0x7fdf2010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf254e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf20102ef0 0x7fdf2010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf254e9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20112060 con 0x7fdf20102570 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf254e9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20111ee0 con 0x7fdf20102ef0 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf254e9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201121e0 con 0x7fdf20100e20 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf1e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20102570 0x7fdf2010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf1effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20100e20 0x7fdf2010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf1effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20100e20 0x7fdf2010e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:35786/0 (socket says 172.21.15.31:35786) 2024-09-16T00:42:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf1effd640 1 -- 172.21.15.31:0/1281297894 learned_addr learned my addr 172.21.15.31:0/1281297894 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf1e7fc640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20102570 0x7fdf2010ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55018/0 (socket says 172.21.15.31:55018) 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.306+0000 7fdf1f7fe640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf20102ef0 0x7fdf2010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf1e7fc640 1 -- 172.21.15.31:0/1281297894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20100e20 msgr2=0x7fdf2010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf1e7fc640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20100e20 0x7fdf2010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf1e7fc640 1 -- 172.21.15.31:0/1281297894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf20102ef0 msgr2=0x7fdf2010ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf1e7fc640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf20102ef0 0x7fdf2010ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf1e7fc640 1 -- 172.21.15.31:0/1281297894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf201a6a50 con 0x7fdf20102570 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf1e7fc640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20102570 0x7fdf2010ea10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d6d0 tx=0x7fdf1400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf07fff640 1 -- 172.21.15.31:0/1281297894 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14017070 con 0x7fdf20102570 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf201a6d40 con 0x7fdf20102570 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf201a7270 con 0x7fdf20102570 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.307+0000 7fdf07fff640 1 -- 172.21.15.31:0/1281297894 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf14009820 con 0x7fdf20102570 2024-09-16T00:42:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.308+0000 7fdf07fff640 1 -- 172.21.15.31:0/1281297894 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14004cd0 con 0x7fdf20102570 2024-09-16T00:42:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.309+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c005290 con 0x7fdf20102570 2024-09-16T00:42:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.311+0000 7fdf07fff640 1 -- 172.21.15.31:0/1281297894 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fdf14005430 con 0x7fdf20102570 2024-09-16T00:42:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.312+0000 7fdf07fff640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf080a32c0 0x7fdf080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.312+0000 7fdf1effd640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf080a32c0 0x7fdf080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.312+0000 7fdf07fff640 1 -- 172.21.15.31:0/1281297894 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fdf140cf9e0 con 0x7fdf20102570 2024-09-16T00:42:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.313+0000 7fdf1effd640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf080a32c0 0x7fdf080a5780 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005770 tx=0x7fdf10005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.317+0000 7fdf07fff640 1 -- 172.21.15.31:0/1281297894 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf1409bd80 con 0x7fdf20102570 2024-09-16T00:42:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.551+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf0c005580 con 0x7fdf20102570 2024-09-16T00:42:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.552+0000 7fdf07fff640 1 -- 172.21.15.31:0/1281297894 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fdf140a0080 con 0x7fdf20102570 2024-09-16T00:42:00.554 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:42:00.554 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:42:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:42:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf080a32c0 msgr2=0x7fdf080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf080a32c0 0x7fdf080a5780 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005770 tx=0x7fdf10005a20 comp rx=0 tx=0).stop 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20102570 msgr2=0x7fdf2010ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20102570 0x7fdf2010ea10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d6d0 tx=0x7fdf1400db90 comp rx=0 tx=0).stop 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 shutdown_connections 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf20102ef0 0x7fdf2010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf20102570 0x7fdf2010ea10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdf080a32c0 0x7fdf080a5780 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 --2- 172.21.15.31:0/1281297894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20100e20 0x7fdf2010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 >> 172.21.15.31:0/1281297894 conn(0x7fdf200fc7d0 msgr2=0x7fdf200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 shutdown_connections 2024-09-16T00:42:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:00.556+0000 7fdf254e9640 1 -- 172.21.15.31:0/1281297894 wait complete. 2024-09-16T00:42:01.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:42:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:01 smithi081 ceph-mon[35539]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:42:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:01 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1281297894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:01.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:01 smithi102 ceph-mon[35364]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:42:01.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:01 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1281297894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:01.623 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:01.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:01 smithi031 ceph-mon[30956]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-16T00:42:01.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:01 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1281297894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.136+0000 7fa761e3d640 1 -- 172.21.15.31:0/1034601937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 msgr2=0x7fa75c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.136+0000 7fa761e3d640 1 --2- 172.21.15.31:0/1034601937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 0x7fa75c107b90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa750009930 tx=0x7fa75002a420 comp rx=0 tx=0).stop 2024-09-16T00:42:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.136+0000 7fa761e3d640 1 -- 172.21.15.31:0/1034601937 shutdown_connections 2024-09-16T00:42:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.136+0000 7fa761e3d640 1 --2- 172.21.15.31:0/1034601937 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c1082a0 0x7fa75c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.136+0000 7fa761e3d640 1 --2- 172.21.15.31:0/1034601937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 0x7fa75c107b90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.136+0000 7fa761e3d640 1 --2- 172.21.15.31:0/1034601937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa75c102d30 0x7fa75c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.136+0000 7fa761e3d640 1 -- 172.21.15.31:0/1034601937 >> 172.21.15.31:0/1034601937 conn(0x7fa75c0fc7d0 msgr2=0x7fa75c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.136+0000 7fa761e3d640 1 -- 172.21.15.31:0/1034601937 shutdown_connections 2024-09-16T00:42:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.136+0000 7fa761e3d640 1 -- 172.21.15.31:0/1034601937 wait complete. 2024-09-16T00:42:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.137+0000 7fa761e3d640 1 Processor -- start 2024-09-16T00:42:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.137+0000 7fa761e3d640 1 -- start start 2024-09-16T00:42:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.137+0000 7fa761e3d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c102d30 0x7fa75c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.137+0000 7fa761e3d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 0x7fa75c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.137+0000 7fa761e3d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa75c1082a0 0x7fa75c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa761e3d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1120a0 con 0x7fa75c1036e0 2024-09-16T00:42:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 0x7fa75c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 0x7fa75c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55050/0 (socket says 172.21.15.31:55050) 2024-09-16T00:42:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75affd640 1 -- 172.21.15.31:0/943346253 learned_addr learned my addr 172.21.15.31:0/943346253 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c111f20 con 0x7fa75c102d30 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c102d30 0x7fa75c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c112220 con 0x7fa75c1082a0 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75bfff640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa75c1082a0 0x7fa75c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75bfff640 1 -- 172.21.15.31:0/943346253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c102d30 msgr2=0x7fa75c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75bfff640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c102d30 0x7fa75c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75bfff640 1 -- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 msgr2=0x7fa75c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75bfff640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 0x7fa75c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75bfff640 1 -- 172.21.15.31:0/943346253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c1a68b0 con 0x7fa75c1082a0 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75affd640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 0x7fa75c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.138+0000 7fa75bfff640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa75c1082a0 0x7fa75c101410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa754009fd0 tx=0x7fa75400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.139+0000 7fa758ff9640 1 -- 172.21.15.31:0/943346253 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75400ce20 con 0x7fa75c1082a0 2024-09-16T00:42:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.139+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75c1a6ba0 con 0x7fa75c1082a0 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.139+0000 7fa758ff9640 1 -- 172.21.15.31:0/943346253 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa754005020 con 0x7fa75c1082a0 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.139+0000 7fa758ff9640 1 -- 172.21.15.31:0/943346253 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754010720 con 0x7fa75c1082a0 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.139+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75c1a7130 con 0x7fa75c1082a0 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.141+0000 7fa758ff9640 1 -- 172.21.15.31:0/943346253 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa754005540 con 0x7fa75c1082a0 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.142+0000 7fa758ff9640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa7440a32c0 0x7fa7440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.142+0000 7fa758ff9640 1 -- 172.21.15.31:0/943346253 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa7540d02f0 con 0x7fa75c1082a0 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.142+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa7440a32c0 0x7fa7440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.142+0000 7fa764216740 1 Processor -- start 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.142+0000 7fa764216740 1 -- start start 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa764216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca49b40 0x5590bca48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa764216740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca48a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa764216740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 0x5590bca55760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa764216740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5590bc9ae2f0 con 0x5590bca485d0 2024-09-16T00:42:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa764216740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5590bc955cf0 con 0x5590bca53370 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa764216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5590bc9aa2b0 con 0x5590bca49b40 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca48a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa7440a32c0 0x7fa7440a5780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005fd0 tx=0x7fa74c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca48a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55056/0 (socket says 172.21.15.31:55056) 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75affd640 1 -- 172.21.15.31:0/1726297355 learned_addr learned my addr 172.21.15.31:0/1726297355 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/1726297355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca49b40 0x5590bca48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75affd640 1 -- 172.21.15.31:0/1726297355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca49b40 msgr2=0x5590bca48090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75affd640 1 --2- 172.21.15.31:0/1726297355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca49b40 0x5590bca48090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75affd640 1 -- 172.21.15.31:0/1726297355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 msgr2=0x5590bca55760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75affd640 1 --2- 172.21.15.31:0/1726297355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 0x5590bca55760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.143+0000 7fa75affd640 1 -- 172.21.15.31:0/1726297355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590bc93a3a0 con 0x5590bca485d0 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.144+0000 7fa75affd640 1 --2- 172.21.15.31:0/1726297355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca48a10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa750031540 tx=0x7fa750005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c848ad6e95c50b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.144+0000 7fa741ffb640 1 -- 172.21.15.31:0/1726297355 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750040070 con 0x5590bca485d0 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.144+0000 7fa741ffb640 1 -- 172.21.15.31:0/1726297355 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa750005520 con 0x5590bca485d0 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.144+0000 7fa741ffb640 1 -- 172.21.15.31:0/1726297355 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750035040 con 0x5590bca485d0 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 -- 172.21.15.31:0/1726297355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 msgr2=0x5590bca48a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 --2- 172.21.15.31:0/1726297355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca48a10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa750031540 tx=0x7fa750005d50 comp rx=0 tx=0).stop 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 -- 172.21.15.31:0/1726297355 shutdown_connections 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 --2- 172.21.15.31:0/1726297355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 0x5590bca55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 --2- 172.21.15.31:0/1726297355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca48a10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 --2- 172.21.15.31:0/1726297355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca49b40 0x5590bca48090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 -- 172.21.15.31:0/1726297355 >> 172.21.15.31:0/1726297355 conn(0x5590bca11310 msgr2=0x5590bca44c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 -- 172.21.15.31:0/1726297355 shutdown_connections 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 -- 172.21.15.31:0/1726297355 wait complete. 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.145+0000 7fa764216740 1 -- start start 2024-09-16T00:42:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa764216740 1 Processor -- start 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa764216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 0x5590bca52da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa764216740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca49b40 0x5590bca50050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa764216740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 0x5590bca507e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa764216740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5590bca56010 con 0x5590bca49b40 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa764216740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5590bc969060 con 0x5590bca53370 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa764216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5590bca56190 con 0x5590bca485d0 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa764216740 10 client.0 _ll_register_callbacks cb 0x5590bc6bc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa75affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca49b40 0x5590bca50050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa75affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca49b40 0x5590bca50050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55068/0 (socket says 172.21.15.31:55068) 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa75affd640 1 -- 172.21.15.31:0/1327140625 learned_addr learned my addr 172.21.15.31:0/1327140625 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.146+0000 7fa75bfff640 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 0x5590bca507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 0x5590bca52da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa75affd640 1 -- 172.21.15.31:0/1327140625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 msgr2=0x5590bca52da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa75affd640 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 0x5590bca52da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa75affd640 1 -- 172.21.15.31:0/1327140625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 msgr2=0x5590bca507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa75affd640 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 0x5590bca507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa75affd640 1 -- 172.21.15.31:0/1327140625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590bc71f3c0 con 0x5590bca49b40 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa75bfff640 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 0x5590bca507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa75affd640 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca49b40 0x5590bca50050 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa750009850 tx=0x7fa7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37dde466b236b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa73affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa73affd640 1 -- 172.21.15.31:0/1327140625 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75004d070 con 0x5590bca49b40 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa73affd640 1 -- 172.21.15.31:0/1327140625 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa7500317c0 con 0x5590bca49b40 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.147+0000 7fa764216740 1 -- 172.21.15.31:0/1327140625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa750005520 con 0x5590bca49b40 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.148+0000 7fa73affd640 1 -- 172.21.15.31:0/1327140625 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa750040030 con 0x5590bca49b40 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.148+0000 7fa764216740 10 client.15408 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:02.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.148+0000 7fa764216740 1 -- 172.21.15.31:0/1327140625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5590bca61b20 con 0x5590bca49b40 2024-09-16T00:42:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.148+0000 7fa73affd640 1 -- 172.21.15.31:0/1327140625 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa750059070 con 0x5590bca49b40 2024-09-16T00:42:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.148+0000 7fa73affd640 1 -- 172.21.15.31:0/1327140625 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fa7500457b0 con 0x5590bca49b40 2024-09-16T00:42:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.148+0000 7fa764216740 10 client.15408 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.148+0000 7fa764216740 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.148+0000 7fa764216740 20 client.15408 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.148+0000 7fa764216740 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bca62e70 0x5590bca65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.149+0000 7fa764216740 4 client.15408 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.149+0000 7fa764216740 1 -- 172.21.15.31:0/1327140625 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590bc9bb860 con 0x5590bca62e70 2024-09-16T00:42:02.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.149+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bca62e70 0x5590bca65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.150+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bca62e70 0x5590bca65260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd450f6face2b9f server_cookie=3f0256a5e904c1ea in_seq=0 out_seq=0 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.150+0000 7fa73affd640 10 client.15408 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.150+0000 7fa73affd640 1 -- 172.21.15.31:0/1327140625 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590bc9bb860 con 0x5590bca62e70 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.150+0000 7fa73affd640 10 client.15408 handle_command_reply: tid=0 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 client.15408 shutdown 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 -- 172.21.15.31:0/1327140625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca49b40 msgr2=0x5590bca50050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca49b40 0x5590bca50050 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa750009850 tx=0x7fa7500046c0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 -- 172.21.15.31:0/1327140625 shutdown_connections 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca53370 0x5590bca507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca49b40 0x5590bca50050 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bca62e70 0x5590bca65260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 --2- 172.21.15.31:0/1327140625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 0x5590bca52da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 -- 172.21.15.31:0/1327140625 >> 172.21.15.31:0/1327140625 conn(0x5590bca11310 msgr2=0x5590bca4a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa73affd640 0 client.15408 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.151+0000 7fa764216740 1 -- 172.21.15.31:0/1327140625 shutdown_connections 2024-09-16T00:42:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.152+0000 7fa764216740 1 -- 172.21.15.31:0/1327140625 wait complete. 2024-09-16T00:42:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.152+0000 7fa764216740 20 client.15408 trim_cache size 0 max 16384 2024-09-16T00:42:02.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.177+0000 7fa764216740 1 Processor -- start 2024-09-16T00:42:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.177+0000 7fa764216740 1 -- start start 2024-09-16T00:42:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa764216740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca4b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa764216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca53370 0x5590bca4b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa764216740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 0x5590bca67ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa764216740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1aeee0 con 0x5590bca485d0 2024-09-16T00:42:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa764216740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1b1760 con 0x5590bca44ed0 2024-09-16T00:42:02.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa764216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1af550 con 0x5590bca53370 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca4b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca4b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55076/0 (socket says 172.21.15.31:55076) 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa75b7fe640 1 -- 172.21.15.31:0/3807952718 learned_addr learned my addr 172.21.15.31:0/3807952718 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa75b7fe640 1 -- 172.21.15.31:0/3807952718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca53370 msgr2=0x5590bca4b660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/3807952718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca53370 0x5590bca4b660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.178+0000 7fa75b7fe640 1 -- 172.21.15.31:0/3807952718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 msgr2=0x5590bca67ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.179+0000 7fa75bfff640 1 --2- 172.21.15.31:0/3807952718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 0x5590bca67ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.179+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/3807952718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 0x5590bca67ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.179+0000 7fa75b7fe640 1 -- 172.21.15.31:0/3807952718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590bca45960 con 0x5590bca485d0 2024-09-16T00:42:02.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.179+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/3807952718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca4b120 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa74c011510 tx=0x7fa74c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83208cb30f6a96ca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:02.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.179+0000 7fa7417fa640 1 -- 172.21.15.31:0/3807952718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c01f4f0 con 0x5590bca485d0 2024-09-16T00:42:02.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.179+0000 7fa7417fa640 1 -- 172.21.15.31:0/3807952718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c01fb70 con 0x5590bca485d0 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.179+0000 7fa7417fa640 1 -- 172.21.15.31:0/3807952718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c02abc0 con 0x5590bca485d0 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.180+0000 7fa764216740 1 -- 172.21.15.31:0/3807952718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 msgr2=0x5590bca4b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.180+0000 7fa764216740 1 --2- 172.21.15.31:0/3807952718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca4b120 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa74c011510 tx=0x7fa74c012000 comp rx=0 tx=0).stop 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.180+0000 7fa764216740 1 -- 172.21.15.31:0/3807952718 shutdown_connections 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.180+0000 7fa764216740 1 --2- 172.21.15.31:0/3807952718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 0x5590bca67ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.180+0000 7fa764216740 1 --2- 172.21.15.31:0/3807952718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca53370 0x5590bca4b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.180+0000 7fa764216740 1 --2- 172.21.15.31:0/3807952718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca485d0 0x5590bca4b120 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.180+0000 7fa764216740 1 -- 172.21.15.31:0/3807952718 >> 172.21.15.31:0/3807952718 conn(0x5590bca11310 msgr2=0x5590bca4f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.180+0000 7fa764216740 1 -- 172.21.15.31:0/3807952718 shutdown_connections 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.180+0000 7fa764216740 1 -- 172.21.15.31:0/3807952718 wait complete. 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.181+0000 7fa764216740 1 -- start start 2024-09-16T00:42:02.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.181+0000 7fa764216740 1 Processor -- start 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa764216740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 0x5590bca65e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa764216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 0x5590bca663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 0x5590bca65e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 0x5590bca65e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:58938/0 (socket says 172.21.15.31:58938) 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa764216740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca53370 0x5590bca66910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa75b7fe640 1 -- 172.21.15.31:0/516996547 learned_addr learned my addr 172.21.15.31:0/516996547 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1afc20 con 0x5590bca53370 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa75affd640 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 0x5590bca663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1b1090 con 0x5590bca44ed0 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.182+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1b02f0 con 0x5590bca485d0 2024-09-16T00:42:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa75bfff640 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca53370 0x5590bca66910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa75affd640 1 -- 172.21.15.31:0/516996547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 msgr2=0x5590bca65e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa75affd640 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 0x5590bca65e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa75affd640 1 -- 172.21.15.31:0/516996547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca53370 msgr2=0x5590bca66910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa764216740 10 client.0 _ll_register_callbacks cb 0x7fa74c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa75affd640 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca53370 0x5590bca66910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa75affd640 1 -- 172.21.15.31:0/516996547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590bca44510 con 0x5590bca485d0 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa74c01fb70 con 0x5590bca485d0 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa75bfff640 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca53370 0x5590bca66910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa75affd640 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 0x5590bca663d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5590bca51140 tx=0x7fa750031490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be31893d538ef48f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.183+0000 7fa73affd640 10 client.25069 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa73affd640 1 -- 172.21.15.31:0/516996547 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75004d070 con 0x5590bca485d0 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa73affd640 1 -- 172.21.15.31:0/516996547 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa750045460 con 0x5590bca485d0 2024-09-16T00:42:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa73affd640 1 -- 172.21.15.31:0/516996547 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa750035040 con 0x5590bca485d0 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa764216740 10 client.25069 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=20}) -- 0x5590bcacd9e0 con 0x5590bca485d0 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa73affd640 1 -- 172.21.15.31:0/516996547 <== mon.1 v2:172.21.15.102:3300/0 4 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fa7500456c0 con 0x5590bca485d0 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa764216740 10 client.25069 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa764216740 10 client.25069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa764216740 20 client.25069 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa764216740 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bcacdf40 0x5590bcad0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa764216740 4 client.25069 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.184+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590bc9b5d70 con 0x5590bcacdf40 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.185+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bcacdf40 0x5590bcad0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.185+0000 7fa75b7fe640 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bcacdf40 0x5590bcad0330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e7c02db856a54 server_cookie=3c30d2f8e2132804 in_seq=0 out_seq=0 2024-09-16T00:42:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.185+0000 7fa73affd640 10 client.25069 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:02.191 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:42:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa73affd640 1 -- 172.21.15.31:0/516996547 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590bc9b5d70 con 0x5590bcacdf40 2024-09-16T00:42:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa73affd640 10 client.25069 handle_command_reply: tid=0 2024-09-16T00:42:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 client.25069 shutdown 2024-09-16T00:42:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 msgr2=0x5590bca663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 0x5590bca663d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5590bca51140 tx=0x7fa750031490 comp rx=0 tx=0).stop 2024-09-16T00:42:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 shutdown_connections 2024-09-16T00:42:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bca53370 0x5590bca66910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bcacdf40 0x5590bcad0330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa73affd640 0 client.25069 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bca485d0 0x5590bca663d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 --2- 172.21.15.31:0/516996547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bca44ed0 0x5590bca65e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 >> 172.21.15.31:0/516996547 conn(0x5590bca11310 msgr2=0x5590bca5e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 shutdown_connections 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 1 -- 172.21.15.31:0/516996547 wait complete. 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.186+0000 7fa764216740 20 client.25069 trim_cache size 0 max 16384 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa7440a32c0 msgr2=0x7fa7440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa7440a32c0 0x7fa7440a5780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005fd0 tx=0x7fa74c005850 comp rx=0 tx=0).stop 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa75c1082a0 msgr2=0x7fa75c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa75c1082a0 0x7fa75c101410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa754009fd0 tx=0x7fa75400ef10 comp rx=0 tx=0).stop 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 shutdown_connections 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa7440a32c0 0x7fa7440a5780 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa75c1082a0 0x7fa75c101410 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa75c1036e0 0x7fa75c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 --2- 172.21.15.31:0/943346253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c102d30 0x7fa75c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.187+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 >> 172.21.15.31:0/943346253 conn(0x7fa75c0fc7d0 msgr2=0x7fa75c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.188+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 shutdown_connections 2024-09-16T00:42:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:02.188+0000 7fa761e3d640 1 -- 172.21.15.31:0/943346253 wait complete. 2024-09-16T00:42:03.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:03 smithi081 ceph-mon[35539]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-16T00:42:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:03 smithi102 ceph-mon[35364]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-16T00:42:03.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:03 smithi031 ceph-mon[30956]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-16T00:42:03.887 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:42:03.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:42:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:04.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.754+0000 7f29ac606640 1 -- 172.21.15.31:0/2161985087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a41009b0 msgr2=0x7f29a4100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.754+0000 7f29ac606640 1 --2- 172.21.15.31:0/2161985087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a41009b0 0x7f29a4100d90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009900 tx=0x7f29a002a410 comp rx=0 tx=0).stop 2024-09-16T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.754+0000 7f29ac606640 1 -- 172.21.15.31:0/2161985087 shutdown_connections 2024-09-16T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.754+0000 7f29ac606640 1 --2- 172.21.15.31:0/2161985087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29a410e4d0 0x7f29a41108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.754+0000 7f29ac606640 1 --2- 172.21.15.31:0/2161985087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a4101360 0x7f29a410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.754+0000 7f29ac606640 1 --2- 172.21.15.31:0/2161985087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a41009b0 0x7f29a4100d90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.754+0000 7f29ac606640 1 -- 172.21.15.31:0/2161985087 >> 172.21.15.31:0/2161985087 conn(0x7f29a40fc7d0 msgr2=0x7f29a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.754+0000 7f29ac606640 1 -- 172.21.15.31:0/2161985087 shutdown_connections 2024-09-16T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.754+0000 7f29ac606640 1 -- 172.21.15.31:0/2161985087 wait complete. 2024-09-16T00:42:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29ac606640 1 Processor -- start 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29ac606640 1 -- start start 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29ac606640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29a41009b0 0x7f29a419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29ac606640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a4101360 0x7f29a419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29ac606640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a410e4d0 0x7f29a419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29ac606640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4114210 con 0x7f29a4101360 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29ac606640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4114090 con 0x7f29a410e4d0 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29ac606640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4114390 con 0x7f29a41009b0 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29a9ba1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a4101360 0x7f29a419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29a9ba1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a4101360 0x7f29a419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55102/0 (socket says 172.21.15.31:55102) 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29a9ba1640 1 -- 172.21.15.31:0/3671896413 learned_addr learned my addr 172.21.15.31:0/3671896413 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.755+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29a41009b0 0x7f29a419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29aaba3640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a410e4d0 0x7f29a419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29a9ba1640 1 -- 172.21.15.31:0/3671896413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29a41009b0 msgr2=0x7f29a419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29a41009b0 0x7f29a419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29a9ba1640 1 -- 172.21.15.31:0/3671896413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a410e4d0 msgr2=0x7f29a419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a410e4d0 0x7f29a419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29a9ba1640 1 -- 172.21.15.31:0/3671896413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a419f6d0 con 0x7f29a4101360 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29aaba3640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a410e4d0 0x7f29a419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29a41009b0 0x7f29a419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a4101360 0x7f29a419ac60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f299400d950 tx=0x7f299400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f299b7fe640 1 -- 172.21.15.31:0/3671896413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994017030 con 0x7f29a4101360 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f29ac606640 1 -- 172.21.15.31:0/3671896413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a419f9c0 con 0x7f29a4101360 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.756+0000 7f299b7fe640 1 -- 172.21.15.31:0/3671896413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2994004020 con 0x7f29a4101360 2024-09-16T00:42:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.757+0000 7f299b7fe640 1 -- 172.21.15.31:0/3671896413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29940047a0 con 0x7f29a4101360 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.757+0000 7f29ac606640 1 -- 172.21.15.31:0/3671896413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a41a72a0 con 0x7f29a4101360 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.759+0000 7f29ae9df740 1 Processor -- start 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.759+0000 7f29ae9df740 1 -- start start 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb59f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb5a480 0x55730fb64800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb67130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29ae9df740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55730fabf280 con 0x55730fb64d40 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29ae9df740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55730fa66b10 con 0x55730fb5a480 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29ae9df740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55730fabb240 con 0x55730fb59b60 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29aaba3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb59f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29aaba3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb59f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:35870/0 (socket says 172.21.15.31:35870) 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29aaba3640 1 -- 172.21.15.31:0/3118738264 learned_addr learned my addr 172.21.15.31:0/3118738264 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb5a480 0x55730fb64800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29aaba3640 1 -- 172.21.15.31:0/3118738264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb5a480 msgr2=0x55730fb64800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29aaba3640 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb5a480 0x55730fb64800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f29aaba3640 1 -- 172.21.15.31:0/3118738264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 msgr2=0x55730fb67130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.760+0000 7f299b7fe640 1 -- 172.21.15.31:0/3671896413 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f29940041d0 con 0x7f29a4101360 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.761+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb67130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.761+0000 7f29aaba3640 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb67130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.761+0000 7f29aaba3640 1 -- 172.21.15.31:0/3118738264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55730fa731f0 con 0x55730fb59b60 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.761+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb5a480 0x55730fb64800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.761+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb67130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.762+0000 7f29aaba3640 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb59f40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f299c009f30 tx=0x7f299c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e3d5f3a8579cd43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.762+0000 7f299b7fe640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f29900a32c0 0x7f29900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.762+0000 7f2998ff9640 1 -- 172.21.15.31:0/3118738264 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c016070 con 0x55730fb59b60 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.762+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f29900a32c0 0x7f29900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.762+0000 7f299b7fe640 1 -- 172.21.15.31:0/3671896413 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f29940cf710 con 0x7f29a4101360 2024-09-16T00:42:04.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.762+0000 7f2998ff9640 1 -- 172.21.15.31:0/3118738264 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299c01bb40 con 0x55730fb59b60 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.762+0000 7f2998ff9640 1 -- 172.21.15.31:0/3118738264 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c012cb0 con 0x55730fb59b60 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29ae9df740 1 -- 172.21.15.31:0/3118738264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 msgr2=0x55730fb59f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f29900a32c0 0x7f29900a5780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009230 tx=0x7f29a00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29ae9df740 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb59f40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f299c009f30 tx=0x7f299c010800 comp rx=0 tx=0).stop 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29ae9df740 1 -- 172.21.15.31:0/3118738264 shutdown_connections 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29ae9df740 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb67130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29ae9df740 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb5a480 0x55730fb64800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29ae9df740 1 --2- 172.21.15.31:0/3118738264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb59f40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29ae9df740 1 -- 172.21.15.31:0/3118738264 >> 172.21.15.31:0/3118738264 conn(0x55730fb560f0 msgr2=0x55730fb564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29ae9df740 1 -- 172.21.15.31:0/3118738264 shutdown_connections 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.763+0000 7f29ae9df740 1 -- 172.21.15.31:0/3118738264 wait complete. 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.764+0000 7f29ae9df740 1 -- start start 2024-09-16T00:42:04.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.764+0000 7f29ae9df740 1 Processor -- start 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.764+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb59b60 0x55730fb61c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.764+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb5a480 0x55730fb621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.764+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb64d40 0x55730fb626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29ae9df740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55730fb679b0 con 0x55730fb5a480 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29ae9df740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55730fa79f10 con 0x55730fb59b60 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29ae9df740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55730fb67b30 con 0x55730fb64d40 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29a9ba1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb5a480 0x55730fb621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29ae9df740 10 client.0 _ll_register_callbacks cb 0x55730fa42a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29a9ba1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb5a480 0x55730fb621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55130/0 (socket says 172.21.15.31:55130) 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29a9ba1640 1 -- 172.21.15.31:0/2711122683 learned_addr learned my addr 172.21.15.31:0/2711122683 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb59b60 0x55730fb61c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29aaba3640 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb64d40 0x55730fb626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29a9ba1640 1 -- 172.21.15.31:0/2711122683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb64d40 msgr2=0x55730fb626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb64d40 0x55730fb626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29a9ba1640 1 -- 172.21.15.31:0/2711122683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb59b60 msgr2=0x55730fb61c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb59b60 0x55730fb61c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29a9ba1640 1 -- 172.21.15.31:0/2711122683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55730f8306a0 con 0x55730fb5a480 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb59b60 0x55730fb61c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb5a480 0x55730fb621a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29940a4060 tx=0x7f299409b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6803bdbf7b2f8acb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.765+0000 7f298dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.766+0000 7f298dffb640 1 -- 172.21.15.31:0/2711122683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299409fcd0 con 0x55730fb5a480 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.766+0000 7f29ae9df740 1 -- 172.21.15.31:0/2711122683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f299c01bb40 con 0x55730fb5a480 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.766+0000 7f298dffb640 1 -- 172.21.15.31:0/2711122683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f29940cf400 con 0x55730fb5a480 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.766+0000 7f298dffb640 1 -- 172.21.15.31:0/2711122683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f29940a4590 con 0x55730fb5a480 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.766+0000 7f29ae9df740 10 client.15420 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.766+0000 7f29ae9df740 1 -- 172.21.15.31:0/2711122683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55730fb73610 con 0x55730fb5a480 2024-09-16T00:42:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.766+0000 7f298dffb640 1 -- 172.21.15.31:0/2711122683 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299c01bb40 con 0x55730fb5a480 2024-09-16T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.766+0000 7f298dffb640 1 -- 172.21.15.31:0/2711122683 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f2994097670 con 0x55730fb5a480 2024-09-16T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.767+0000 7f29ae9df740 10 client.15420 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.767+0000 7f29ae9df740 10 client.15420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.767+0000 7f29ae9df740 20 client.15420 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.767+0000 7f29ae9df740 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55730fb74da0 0x55730fb77240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.767+0000 7f29ae9df740 4 client.15420 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:04.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.767+0000 7f29ae9df740 1 -- 172.21.15.31:0/2711122683 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55730facc7f0 con 0x55730fb74da0 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.767+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55730fb74da0 0x55730fb77240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.768+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55730fb74da0 0x55730fb77240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc74bde556eeb70d server_cookie=61872a01db929537 in_seq=0 out_seq=0 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.768+0000 7f298dffb640 10 client.15420 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.768+0000 7f298dffb640 1 -- 172.21.15.31:0/2711122683 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55730facc7f0 con 0x55730fb74da0 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.768+0000 7f298dffb640 10 client.15420 handle_command_reply: tid=0 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.768+0000 7f29ae9df740 1 client.15420 shutdown 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 -- 172.21.15.31:0/2711122683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb5a480 msgr2=0x55730fb621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb5a480 0x55730fb621a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29940a4060 tx=0x7f299409b520 comp rx=0 tx=0).stop 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 -- 172.21.15.31:0/2711122683 shutdown_connections 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55730fb74da0 0x55730fb77240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb64d40 0x55730fb626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb5a480 0x55730fb621a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 --2- 172.21.15.31:0/2711122683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb59b60 0x55730fb61c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 -- 172.21.15.31:0/2711122683 >> 172.21.15.31:0/2711122683 conn(0x55730fb560f0 msgr2=0x55730fb55270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f298dffb640 0 client.15420 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 -- 172.21.15.31:0/2711122683 shutdown_connections 2024-09-16T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 1 -- 172.21.15.31:0/2711122683 wait complete. 2024-09-16T00:42:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.769+0000 7f29ae9df740 20 client.15420 trim_cache size 0 max 16384 2024-09-16T00:42:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.803+0000 7f29ae9df740 1 Processor -- start 2024-09-16T00:42:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.803+0000 7f29ae9df740 1 -- start start 2024-09-16T00:42:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.803+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.803+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb7cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb60d50 0x55730fb7d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29a9ba1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb7cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29a9ba1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb7cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55142/0 (socket says 172.21.15.31:55142) 2024-09-16T00:42:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29a9ba1640 1 -- 172.21.15.31:0/234592569 learned_addr learned my addr 172.21.15.31:0/234592569 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29ae9df740 1 -- 172.21.15.31:0/234592569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f29a41aab60 con 0x55730fb64d40 2024-09-16T00:42:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29ae9df740 1 -- 172.21.15.31:0/234592569 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f29a41ad3e0 con 0x55730fb60d50 2024-09-16T00:42:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29ae9df740 1 -- 172.21.15.31:0/234592569 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f29a41ab1d0 con 0x55730fb59b60 2024-09-16T00:42:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29aaba3640 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb60d50 0x55730fb7d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29a9ba1640 1 -- 172.21.15.31:0/234592569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 msgr2=0x55730fb5bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb5bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29a9ba1640 1 -- 172.21.15.31:0/234592569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb60d50 msgr2=0x55730fb7d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb60d50 0x55730fb7d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29a9ba1640 1 -- 172.21.15.31:0/234592569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55730fb601b0 con 0x55730fb64d40 2024-09-16T00:42:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.804+0000 7f29aaba3640 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb60d50 0x55730fb7d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.805+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb7cb50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2994099bd0 tx=0x7f299409b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d5d704e8d97582 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.805+0000 7f298ffff640 1 -- 172.21.15.31:0/234592569 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299409fcd0 con 0x55730fb64d40 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.805+0000 7f298ffff640 1 -- 172.21.15.31:0/234592569 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29940cf400 con 0x55730fb64d40 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.805+0000 7f298ffff640 1 -- 172.21.15.31:0/234592569 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299409aff0 con 0x55730fb64d40 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.806+0000 7f29ae9df740 1 -- 172.21.15.31:0/234592569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 msgr2=0x55730fb7cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.806+0000 7f29ae9df740 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb7cb50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2994099bd0 tx=0x7f299409b590 comp rx=0 tx=0).stop 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.806+0000 7f29ae9df740 1 -- 172.21.15.31:0/234592569 shutdown_connections 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.806+0000 7f29ae9df740 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb60d50 0x55730fb7d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.806+0000 7f29ae9df740 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb64d40 0x55730fb7cb50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.806+0000 7f29ae9df740 1 --2- 172.21.15.31:0/234592569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb59b60 0x55730fb5bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.806+0000 7f29ae9df740 1 -- 172.21.15.31:0/234592569 >> 172.21.15.31:0/234592569 conn(0x55730fb560f0 msgr2=0x55730fb5b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.806+0000 7f29ae9df740 1 -- 172.21.15.31:0/234592569 shutdown_connections 2024-09-16T00:42:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.806+0000 7f29ae9df740 1 -- 172.21.15.31:0/234592569 wait complete. 2024-09-16T00:42:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.807+0000 7f29ae9df740 1 -- start start 2024-09-16T00:42:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.807+0000 7f29ae9df740 1 Processor -- start 2024-09-16T00:42:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.807+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb59b60 0x55730fb5ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.807+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb60d50 0x55730fb5f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29ae9df740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb64d40 0x55730fb5f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29aa3a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb59b60 0x55730fb5ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29aa3a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb59b60 0x55730fb5ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55146/0 (socket says 172.21.15.31:55146) 2024-09-16T00:42:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29aa3a2640 1 -- 172.21.15.31:0/370644769 learned_addr learned my addr 172.21.15.31:0/370644769 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f29a41ab8a0 con 0x55730fb59b60 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f29a41acd10 con 0x55730fb64d40 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f29a41abf70 con 0x55730fb60d50 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29ae9df740 10 client.0 _ll_register_callbacks cb 0x7f2994097cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb60d50 0x55730fb5f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29aaba3640 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb64d40 0x55730fb5f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29a9ba1640 1 -- 172.21.15.31:0/370644769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb64d40 msgr2=0x55730fb5f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb64d40 0x55730fb5f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29a9ba1640 1 -- 172.21.15.31:0/370644769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb59b60 msgr2=0x55730fb5ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb59b60 0x55730fb5ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29a9ba1640 1 -- 172.21.15.31:0/370644769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55730fbdec80 con 0x55730fb60d50 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29aaba3640 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb64d40 0x55730fb5f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f29a9ba1640 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb60d50 0x55730fb5f320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55730fb604c0 tx=0x7f2994005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4db65be6c7beda70 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f298dffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.808+0000 7f298dffb640 1 -- 172.21.15.31:0/370644769 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299409fcd0 con 0x55730fb60d50 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.809+0000 7f298dffb640 1 -- 172.21.15.31:0/370644769 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f299409ad70 con 0x55730fb60d50 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.809+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29940cf400 con 0x55730fb60d50 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.809+0000 7f298dffb640 1 -- 172.21.15.31:0/370644769 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f29940ba2a0 con 0x55730fb60d50 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.809+0000 7f29ae9df740 10 client.25087 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.809+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55730fbdffa0 con 0x55730fb60d50 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.809+0000 7f298dffb640 1 -- 172.21.15.31:0/370644769 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29940cf400 con 0x55730fb60d50 2024-09-16T00:42:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.810+0000 7f298dffb640 1 -- 172.21.15.31:0/370644769 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f299409a9a0 con 0x55730fb60d50 2024-09-16T00:42:04.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.810+0000 7f29ae9df740 10 client.25087 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:04.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.810+0000 7f29ae9df740 10 client.25087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:04.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.810+0000 7f29ae9df740 20 client.25087 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:04.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.810+0000 7f29ae9df740 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55730fbe04b0 0x55730fbe28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:04.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.810+0000 7f29ae9df740 4 client.25087 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:42:04.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.810+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55730fac6d00 con 0x55730fbe04b0 2024-09-16T00:42:04.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.810+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55730fbe04b0 0x55730fbe28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:04.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.811+0000 7f29aa3a2640 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55730fbe04b0 0x55730fbe28d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e62292563a12c3 server_cookie=8e5a9fd405fe0cd8 in_seq=0 out_seq=0 2024-09-16T00:42:04.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.811+0000 7f298dffb640 10 client.25087 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:04.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.812+0000 7f298dffb640 1 -- 172.21.15.31:0/370644769 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55730fac6d00 con 0x55730fbe04b0 2024-09-16T00:42:04.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.812+0000 7f298dffb640 10 client.25087 handle_command_reply: tid=0 2024-09-16T00:42:04.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.812+0000 7f29ae9df740 1 client.25087 shutdown 2024-09-16T00:42:04.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.812+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb60d50 msgr2=0x55730fb5f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.812+0000 7f29ae9df740 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb60d50 0x55730fb5f320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55730fb604c0 tx=0x7f2994005dc0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 shutdown_connections 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f29ae9df740 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55730fb64d40 0x55730fb5f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f29ae9df740 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55730fbe04b0 0x55730fbe28d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f29ae9df740 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55730fb60d50 0x55730fb5f320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f298dffb640 0 client.25087 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f29ae9df740 1 --2- 172.21.15.31:0/370644769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55730fb59b60 0x55730fb5ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 >> 172.21.15.31:0/370644769 conn(0x55730fb560f0 msgr2=0x55730fb62440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 shutdown_connections 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f29ae9df740 1 -- 172.21.15.31:0/370644769 wait complete. 2024-09-16T00:42:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.814+0000 7f29ae9df740 20 client.25087 trim_cache size 0 max 16384 2024-09-16T00:42:04.818 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:42:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:42:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "eb3be128-b52a-4e83-a833-4ecdba8213ae", 2024-09-16T00:42:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:42:04.818 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:42:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 -- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f29900a32c0 msgr2=0x7f29900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f29900a32c0 0x7f29900a5780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009230 tx=0x7f29a00058d0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 -- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a4101360 msgr2=0x7f29a419ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a4101360 0x7f29a419ac60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f299400d950 tx=0x7f299400de10 comp rx=0 tx=0).stop 2024-09-16T00:42:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 -- 172.21.15.31:0/3671896413 shutdown_connections 2024-09-16T00:42:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a410e4d0 0x7f29a419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29a4101360 0x7f29a419ac60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f29900a32c0 0x7f29900a5780 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 --2- 172.21.15.31:0/3671896413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29a41009b0 0x7f29a419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 -- 172.21.15.31:0/3671896413 >> 172.21.15.31:0/3671896413 conn(0x7f29a40fc7d0 msgr2=0x7f29a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 -- 172.21.15.31:0/3671896413 shutdown_connections 2024-09-16T00:42:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:04.815+0000 7f29ac606640 1 -- 172.21.15.31:0/3671896413 wait complete. 2024-09-16T00:42:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:05 smithi081 ceph-mon[35539]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T00:42:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:05 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:42:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:05 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:05 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:42:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:05 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:05 smithi102 ceph-mon[35364]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T00:42:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:05 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:42:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:05 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:05 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:42:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:05 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:05.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:05 smithi031 ceph-mon[30956]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T00:42:05.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:05 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:42:05.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:05 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:05.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:05 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:42:05.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:05 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:05.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:42:06.154 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.749+0000 7f5914f89640 1 -- 172.21.15.31:0/2958790448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910102d30 msgr2=0x7f5910103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.749+0000 7f5914f89640 1 --2- 172.21.15.31:0/2958790448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910102d30 0x7f5910103110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f59000099b0 tx=0x7f590002e850 comp rx=0 tx=0).stop 2024-09-16T00:42:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.750+0000 7f5914f89640 1 -- 172.21.15.31:0/2958790448 shutdown_connections 2024-09-16T00:42:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.750+0000 7f5914f89640 1 --2- 172.21.15.31:0/2958790448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59101082a0 0x7f591010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.750+0000 7f5914f89640 1 --2- 172.21.15.31:0/2958790448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59101036e0 0x7f5910107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.750+0000 7f5914f89640 1 --2- 172.21.15.31:0/2958790448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5910102d30 0x7f5910103110 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.750+0000 7f5914f89640 1 -- 172.21.15.31:0/2958790448 >> 172.21.15.31:0/2958790448 conn(0x7f59100fc7d0 msgr2=0x7f59100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.750+0000 7f5914f89640 1 -- 172.21.15.31:0/2958790448 shutdown_connections 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.750+0000 7f5914f89640 1 -- 172.21.15.31:0/2958790448 wait complete. 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.751+0000 7f5914f89640 1 Processor -- start 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.751+0000 7f5914f89640 1 -- start start 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.751+0000 7f5914f89640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5910102d30 0x7f59101963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.751+0000 7f5914f89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59101036e0 0x7f59101968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.751+0000 7f5914f89640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59101082a0 0x7f591019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59101036e0 0x7f59101968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f5914f89640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f591010fe90 con 0x7f59101036e0 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f5914f89640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f591010fd10 con 0x7f59101082a0 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f5914f89640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5910110010 con 0x7f5910102d30 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59101036e0 0x7f59101968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55160/0 (socket says 172.21.15.31:55160) 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590dd9b640 1 -- 172.21.15.31:0/552796787 learned_addr learned my addr 172.21.15.31:0/552796787 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590e59c640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5910102d30 0x7f59101963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590dd9b640 1 -- 172.21.15.31:0/552796787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5910102d30 msgr2=0x7f59101963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590dd9b640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5910102d30 0x7f59101963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590dd9b640 1 -- 172.21.15.31:0/552796787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59101082a0 msgr2=0x7f591019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590dd9b640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59101082a0 0x7f591019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590dd9b640 1 -- 172.21.15.31:0/552796787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591019b360 con 0x7f59101036e0 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.752+0000 7f590dd9b640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59101036e0 0x7f59101968f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f590400d950 tx=0x7f590400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.753+0000 7f58ff7fe640 1 -- 172.21.15.31:0/552796787 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904017070 con 0x7f59101036e0 2024-09-16T00:42:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.753+0000 7f58ff7fe640 1 -- 172.21.15.31:0/552796787 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5904009c30 con 0x7f59101036e0 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.753+0000 7f58ff7fe640 1 -- 172.21.15.31:0/552796787 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59040047a0 con 0x7f59101036e0 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.753+0000 7f5914f89640 1 -- 172.21.15.31:0/552796787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591019b650 con 0x7f59101036e0 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.753+0000 7f5914f89640 1 -- 172.21.15.31:0/552796787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59101a2f30 con 0x7f59101036e0 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.756+0000 7f5917362740 1 Processor -- start 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f5917362740 1 -- start start 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f58ff7fe640 1 -- 172.21.15.31:0/552796787 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5904004020 con 0x7f59101036e0 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f5917362740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e08036f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f5917362740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08036fdc0 0x55e08037a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f5917362740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037a680 0x55e08037ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f5917362740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0802d52e0 con 0x55e08036f4a0 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f5917362740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e08027cce0 con 0x55e08036fdc0 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f5917362740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e0802d12a0 con 0x55e08037a680 2024-09-16T00:42:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f590dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037a680 0x55e08037ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f590dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037a680 0x55e08037ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:35920/0 (socket says 172.21.15.31:35920) 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f590dd9b640 1 -- 172.21.15.31:0/1830559080 learned_addr learned my addr 172.21.15.31:0/1830559080 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f590e59c640 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08036fdc0 0x55e08037a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.757+0000 7f590ed9d640 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e08036f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f590dd9b640 1 -- 172.21.15.31:0/1830559080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08036fdc0 msgr2=0x55e08037a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f590dd9b640 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08036fdc0 0x55e08037a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f590dd9b640 1 -- 172.21.15.31:0/1830559080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 msgr2=0x55e08036f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f590dd9b640 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e08036f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f590dd9b640 1 -- 172.21.15.31:0/1830559080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e080261440 con 0x55e08037a680 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f590ed9d640 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e08036f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f590dd9b640 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037a680 0x55e08037ca70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59040d5740 tx=0x7f59040d5c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b667f3246b7abe58 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f58fcff9640 1 -- 172.21.15.31:0/1830559080 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59040e2070 con 0x55e08037a680 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f58fcff9640 1 -- 172.21.15.31:0/1830559080 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59040d5e50 con 0x55e08037a680 2024-09-16T00:42:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.758+0000 7f58fcff9640 1 -- 172.21.15.31:0/1830559080 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59040d1650 con 0x55e08037a680 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.759+0000 7f5917362740 1 -- 172.21.15.31:0/1830559080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037a680 msgr2=0x55e08037ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.759+0000 7f5917362740 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037a680 0x55e08037ca70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59040d5740 tx=0x7f59040d5c00 comp rx=0 tx=0).stop 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.759+0000 7f5917362740 1 -- 172.21.15.31:0/1830559080 shutdown_connections 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.759+0000 7f5917362740 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037a680 0x55e08037ca70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.759+0000 7f5917362740 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08036fdc0 0x55e08037a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.759+0000 7f5917362740 1 --2- 172.21.15.31:0/1830559080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e08036f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.759+0000 7f5917362740 1 -- 172.21.15.31:0/1830559080 >> 172.21.15.31:0/1830559080 conn(0x55e08036ba30 msgr2=0x55e08036be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.760+0000 7f5917362740 1 -- 172.21.15.31:0/1830559080 shutdown_connections 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.760+0000 7f5917362740 1 -- 172.21.15.31:0/1830559080 wait complete. 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.760+0000 7f58ff7fe640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58f80a3250 0x7f58f80a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.760+0000 7f5917362740 1 -- start start 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.760+0000 7f590ed9d640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58f80a3250 0x7f58f80a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.760+0000 7f58ff7fe640 1 -- 172.21.15.31:0/552796787 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f59040cf6c0 con 0x7f59101036e0 2024-09-16T00:42:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.761+0000 7f590ed9d640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58f80a3250 0x7f58f80a5710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5908005c40 tx=0x7f5908009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.761+0000 7f5917362740 1 Processor -- start 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.761+0000 7f5917362740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080379f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.761+0000 7f5917362740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036fdc0 0x55e080377210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.761+0000 7f5917362740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080377750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.761+0000 7f5917362740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e08037d0d0 con 0x55e08036f4a0 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.761+0000 7f5917362740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e080290050 con 0x55e08037a680 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.761+0000 7f5917362740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e08037d250 con 0x55e08036fdc0 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.761+0000 7f5917362740 10 client.0 _ll_register_callbacks cb 0x55e0802d0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080379f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080377750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036fdc0 0x55e080377210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080377750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:59042/0 (socket says 172.21.15.31:59042) 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590ed9d640 1 -- 172.21.15.31:0/2248910366 learned_addr learned my addr 172.21.15.31:0/2248910366 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590dd9b640 1 -- 172.21.15.31:0/2248910366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 msgr2=0x55e080377750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590dd9b640 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080377750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590dd9b640 1 -- 172.21.15.31:0/2248910366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 msgr2=0x55e080379f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590dd9b640 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080379f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590dd9b640 1 -- 172.21.15.31:0/2248910366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0800463c0 con 0x55e08036fdc0 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f590dd9b640 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036fdc0 0x55e080377210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f590401b000 tx=0x7f59040d7e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f29f675db41fe81 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f58f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.762+0000 7f58f5ffb640 1 -- 172.21.15.31:0/2248910366 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904097b70 con 0x55e08036fdc0 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f58f5ffb640 1 -- 172.21.15.31:0/2248910366 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f59040cf370 con 0x55e08036fdc0 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f5917362740 1 -- 172.21.15.31:0/2248910366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59040d5e50 con 0x55e08036fdc0 2024-09-16T00:42:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f58f5ffb640 1 -- 172.21.15.31:0/2248910366 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f590409b230 con 0x55e08036fdc0 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f5917362740 10 client.25099 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f5917362740 1 -- 172.21.15.31:0/2248910366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e08037d0d0 con 0x55e08036fdc0 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f58f5ffb640 1 -- 172.21.15.31:0/2248910366 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59040d5e50 con 0x55e08036fdc0 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f58f5ffb640 1 -- 172.21.15.31:0/2248910366 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f5904023060 con 0x55e08036fdc0 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f5917362740 10 client.25099 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f5917362740 10 client.25099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f5917362740 20 client.25099 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f5917362740 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55e080389c20 0x55e08038c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f5917362740 4 client.25099 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.763+0000 7f5917362740 1 -- 172.21.15.31:0/2248910366 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0802e2850 con 0x55e080389c20 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.764+0000 7f590e59c640 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55e080389c20 0x55e08038c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.764+0000 7f590e59c640 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55e080389c20 0x55e08038c010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa32b1ac43b0190 server_cookie=f45f2799d4c90d51 in_seq=0 out_seq=0 2024-09-16T00:42:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.764+0000 7f58f5ffb640 10 client.25099 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f58f5ffb640 1 -- 172.21.15.31:0/2248910366 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0802e2850 con 0x55e080389c20 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f58f5ffb640 10 client.25099 handle_command_reply: tid=0 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 client.25099 shutdown 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 -- 172.21.15.31:0/2248910366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036fdc0 msgr2=0x55e080377210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036fdc0 0x55e080377210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f590401b000 tx=0x7f59040d7e10 comp rx=0 tx=0).stop 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 -- 172.21.15.31:0/2248910366 shutdown_connections 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080377750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55e080389c20 0x55e08038c010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036fdc0 0x55e080377210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 --2- 172.21.15.31:0/2248910366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080379f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f58f5ffb640 0 client.25099 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 -- 172.21.15.31:0/2248910366 >> 172.21.15.31:0/2248910366 conn(0x55e08036ba30 msgr2=0x55e080371760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.765+0000 7f5917362740 1 -- 172.21.15.31:0/2248910366 shutdown_connections 2024-09-16T00:42:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.766+0000 7f5917362740 1 -- 172.21.15.31:0/2248910366 wait complete. 2024-09-16T00:42:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.766+0000 7f5917362740 20 client.25099 trim_cache size 0 max 16384 2024-09-16T00:42:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f5917362740 1 Processor -- start 2024-09-16T00:42:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f5917362740 1 -- start start 2024-09-16T00:42:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f5917362740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036f4a0 0x55e080386390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f5917362740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036fdc0 0x55e0803764b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f5917362740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e0803917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f5917362740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f59101aac60 con 0x55e08036fdc0 2024-09-16T00:42:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f5917362740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f59101ad4e0 con 0x55e08037a680 2024-09-16T00:42:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f5917362740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f59101ab2d0 con 0x55e08036f4a0 2024-09-16T00:42:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f590dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036fdc0 0x55e0803764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f590dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036fdc0 0x55e0803764b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55194/0 (socket says 172.21.15.31:55194) 2024-09-16T00:42:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.783+0000 7f590dd9b640 1 -- 172.21.15.31:0/1285920439 learned_addr learned my addr 172.21.15.31:0/1285920439 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f590ed9d640 1 --2- 172.21.15.31:0/1285920439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e0803917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f590e59c640 1 --2- 172.21.15.31:0/1285920439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036f4a0 0x55e080386390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f590dd9b640 1 -- 172.21.15.31:0/1285920439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036f4a0 msgr2=0x55e080386390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f590dd9b640 1 --2- 172.21.15.31:0/1285920439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036f4a0 0x55e080386390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f590dd9b640 1 -- 172.21.15.31:0/1285920439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 msgr2=0x55e0803917f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f590dd9b640 1 --2- 172.21.15.31:0/1285920439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e0803917f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f590dd9b640 1 -- 172.21.15.31:0/1285920439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0803796a0 con 0x55e08036fdc0 2024-09-16T00:42:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f590dd9b640 1 --2- 172.21.15.31:0/1285920439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036fdc0 0x55e0803764b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5904005fd0 tx=0x7f5904005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322cafe5deee4140 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f58f7fff640 1 -- 172.21.15.31:0/1285920439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590409b410 con 0x55e08036fdc0 2024-09-16T00:42:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f58f7fff640 1 -- 172.21.15.31:0/1285920439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59040d5e50 con 0x55e08036fdc0 2024-09-16T00:42:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.784+0000 7f58f7fff640 1 -- 172.21.15.31:0/1285920439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590409b6a0 con 0x55e08036fdc0 2024-09-16T00:42:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.785+0000 7f5917362740 1 -- 172.21.15.31:0/1285920439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036fdc0 msgr2=0x55e0803764b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.785+0000 7f5917362740 1 --2- 172.21.15.31:0/1285920439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036fdc0 0x55e0803764b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5904005fd0 tx=0x7f5904005130 comp rx=0 tx=0).stop 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.785+0000 7f5917362740 1 -- 172.21.15.31:0/1285920439 shutdown_connections 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.785+0000 7f5917362740 1 --2- 172.21.15.31:0/1285920439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e0803917f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.785+0000 7f5917362740 1 --2- 172.21.15.31:0/1285920439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036fdc0 0x55e0803764b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.785+0000 7f5917362740 1 --2- 172.21.15.31:0/1285920439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08036f4a0 0x55e080386390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.785+0000 7f5917362740 1 -- 172.21.15.31:0/1285920439 >> 172.21.15.31:0/1285920439 conn(0x55e08036ba30 msgr2=0x55e08036ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.785+0000 7f5917362740 1 -- 172.21.15.31:0/1285920439 shutdown_connections 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.785+0000 7f5917362740 1 -- 172.21.15.31:0/1285920439 wait complete. 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f5917362740 1 -- start start 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f5917362740 1 Processor -- start 2024-09-16T00:42:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f5917362740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080382500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f5917362740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080382a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f5917362740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037f4d0 0x55e08037f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f5917362740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f59101ab9a0 con 0x55e08036f4a0 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f5917362740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f59101ace10 con 0x55e08037a680 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f5917362740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f59101ac070 con 0x55e08037f4d0 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f5917362740 10 client.0 _ll_register_callbacks cb 0x7f59040e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f590e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080382500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f590e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080382500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55208/0 (socket says 172.21.15.31:55208) 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f590e59c640 1 -- 172.21.15.31:0/775150556 learned_addr learned my addr 172.21.15.31:0/775150556 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.786+0000 7f590dd9b640 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080382a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f590ed9d640 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037f4d0 0x55e08037f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f590dd9b640 1 -- 172.21.15.31:0/775150556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037f4d0 msgr2=0x55e08037f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f590dd9b640 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037f4d0 0x55e08037f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f590dd9b640 1 -- 172.21.15.31:0/775150556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 msgr2=0x55e080382500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f590dd9b640 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080382500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f590dd9b640 1 -- 172.21.15.31:0/775150556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e08037fe90 con 0x55e08037a680 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f590e59c640 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080382500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f590dd9b640 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080382a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55e08037cae0 tx=0x7f59040051a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f1a98ca39e14299 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f58f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f58f5ffb640 1 -- 172.21.15.31:0/775150556 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590409b6a0 con 0x55e08037a680 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f58f5ffb640 1 -- 172.21.15.31:0/775150556 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f5904097f40 con 0x55e08037a680 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.787+0000 7f5917362740 1 -- 172.21.15.31:0/775150556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59040d5e50 con 0x55e08037a680 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f58f5ffb640 1 -- 172.21.15.31:0/775150556 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f590409b410 con 0x55e08037a680 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f5917362740 10 client.25148 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f5917362740 1 -- 172.21.15.31:0/775150556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e08037fe90 con 0x55e08037a680 2024-09-16T00:42:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f58f5ffb640 1 -- 172.21.15.31:0/775150556 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5904014020 con 0x55e08037a680 2024-09-16T00:42:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f58f5ffb640 1 -- 172.21.15.31:0/775150556 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f59040ba0f0 con 0x55e08037a680 2024-09-16T00:42:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f5917362740 10 client.25148 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f5917362740 10 client.25148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f5917362740 20 client.25148 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f5917362740 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55e0803f55f0 0x55e0803f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f5917362740 4 client.25148 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.788+0000 7f5917362740 1 -- 172.21.15.31:0/775150556 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f59040d5e50 con 0x55e0803f55f0 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f590e59c640 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55e0803f55f0 0x55e0803f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f58f5ffb640 10 client.25148 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f590e59c640 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55e0803f55f0 0x55e0803f79e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff5379e905879209 server_cookie=6ef247f20cdb5c00 in_seq=0 out_seq=0 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f58f5ffb640 1 -- 172.21.15.31:0/775150556 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f59040d5e50 con 0x55e0803f55f0 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f58f5ffb640 10 client.25148 handle_command_reply: tid=0 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f5917362740 1 client.25148 shutdown 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f5917362740 1 -- 172.21.15.31:0/775150556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 msgr2=0x55e080382a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f5917362740 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080382a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55e08037cae0 tx=0x7f59040051a0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f5917362740 1 -- 172.21.15.31:0/775150556 shutdown_connections 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.789+0000 7f5917362740 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55e0803f55f0 0x55e0803f79e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.790+0000 7f58f5ffb640 0 client.25148 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.790+0000 7f5917362740 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08037f4d0 0x55e08037f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.790+0000 7f5917362740 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e08037a680 0x55e080382a40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.790+0000 7f5917362740 1 --2- 172.21.15.31:0/775150556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e08036f4a0 0x55e080382500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.790+0000 7f5917362740 1 -- 172.21.15.31:0/775150556 >> 172.21.15.31:0/775150556 conn(0x55e08036ba30 msgr2=0x55e080370ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.790+0000 7f5917362740 1 -- 172.21.15.31:0/775150556 shutdown_connections 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.790+0000 7f5917362740 1 -- 172.21.15.31:0/775150556 wait complete. 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.790+0000 7f5917362740 20 client.25148 trim_cache size 0 max 16384 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:42:06.794 INFO:teuthology.orchestra.run.smithi031.stdout: "eb3be128-b52a-4e83-a833-4ecdba8213ae": { 2024-09-16T00:42:06.795 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:42:06.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "eb3be128-b52a-4e83-a833-4ecdba8213ae", 2024-09-16T00:42:06.795 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:42:06.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:42:06.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:42:06.795 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 -- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58f80a3250 msgr2=0x7f58f80a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58f80a3250 0x7f58f80a5710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5908005c40 tx=0x7f5908009560 comp rx=0 tx=0).stop 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 -- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59101036e0 msgr2=0x7f59101968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59101036e0 0x7f59101968f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f590400d950 tx=0x7f590400de10 comp rx=0 tx=0).stop 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 -- 172.21.15.31:0/552796787 shutdown_connections 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59101082a0 0x7f591019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59101036e0 0x7f59101968f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f58f80a3250 0x7f58f80a5710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 --2- 172.21.15.31:0/552796787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5910102d30 0x7f59101963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 -- 172.21.15.31:0/552796787 >> 172.21.15.31:0/552796787 conn(0x7f59100fc7d0 msgr2=0x7f5910105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 -- 172.21.15.31:0/552796787 shutdown_connections 2024-09-16T00:42:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:06.791+0000 7f5914f89640 1 -- 172.21.15.31:0/552796787 wait complete. 2024-09-16T00:42:07.455 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:07 smithi031 ceph-mon[30956]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-16T00:42:07.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:07 smithi081 ceph-mon[35539]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-16T00:42:07.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:07 smithi102 ceph-mon[35364]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-16T00:42:07.639 INFO:tasks.cephfs.filesystem:scrub status for tag:eb3be128-b52a-4e83-a833-4ecdba8213ae - {'path': '/', 'tag': 'eb3be128-b52a-4e83-a833-4ecdba8213ae', 'options': 'recursive,force'} 2024-09-16T00:42:08.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:08 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:42:08.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:08 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:42:08.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:08 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:42:09.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:09 smithi081 ceph-mon[35539]: pgmap v386: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T00:42:09.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:09 smithi102 ceph-mon[35364]: pgmap v386: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T00:42:09.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:09 smithi031 ceph-mon[30956]: pgmap v386: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T00:42:11.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:11 smithi081 ceph-mon[35539]: pgmap v387: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T00:42:11.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:11 smithi102 ceph-mon[35364]: pgmap v387: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T00:42:11.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:11 smithi031 ceph-mon[30956]: pgmap v387: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T00:42:13.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:13 smithi081 ceph-mon[35539]: pgmap v388: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-09-16T00:42:13.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:13 smithi102 ceph-mon[35364]: pgmap v388: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-09-16T00:42:13.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:13 smithi031 ceph-mon[30956]: pgmap v388: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-09-16T00:42:14.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:42:14.673 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:15.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.168+0000 7ff26696f640 1 -- 172.21.15.31:0/2844426846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff260102d30 msgr2=0x7ff260103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:15.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.168+0000 7ff26696f640 1 --2- 172.21.15.31:0/2844426846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff260102d30 0x7ff260103110 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff250009930 tx=0x7ff25002a420 comp rx=0 tx=0).stop 2024-09-16T00:42:15.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.168+0000 7ff26696f640 1 -- 172.21.15.31:0/2844426846 shutdown_connections 2024-09-16T00:42:15.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.168+0000 7ff26696f640 1 --2- 172.21.15.31:0/2844426846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2601082a0 0x7ff26010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:15.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.168+0000 7ff26696f640 1 --2- 172.21.15.31:0/2844426846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2601036e0 0x7ff260107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:15.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.168+0000 7ff26696f640 1 --2- 172.21.15.31:0/2844426846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff260102d30 0x7ff260103110 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:15.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.168+0000 7ff26696f640 1 -- 172.21.15.31:0/2844426846 >> 172.21.15.31:0/2844426846 conn(0x7ff2600fc7d0 msgr2=0x7ff2600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:15.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.168+0000 7ff26696f640 1 -- 172.21.15.31:0/2844426846 shutdown_connections 2024-09-16T00:42:15.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.168+0000 7ff26696f640 1 -- 172.21.15.31:0/2844426846 wait complete. 2024-09-16T00:42:15.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.169+0000 7ff26696f640 1 Processor -- start 2024-09-16T00:42:15.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.169+0000 7ff26696f640 1 -- start start 2024-09-16T00:42:15.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff26696f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff260102d30 0x7ff2601985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:15.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff26696f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2601036e0 0x7ff260198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:15.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff26696f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2601082a0 0x7ff26019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:15.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff26696f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2601120a0 con 0x7ff2601082a0 2024-09-16T00:42:15.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff26696f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff260111f20 con 0x7ff260102d30 2024-09-16T00:42:15.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff26696f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff260112220 con 0x7ff2601036e0 2024-09-16T00:42:15.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff25ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff260102d30 0x7ff2601985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff264f0c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2601082a0 0x7ff26019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff25ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff260102d30 0x7ff2601985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:50992/0 (socket says 172.21.15.31:50992) 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff25ffff640 1 -- 172.21.15.31:0/1659835516 learned_addr learned my addr 172.21.15.31:0/1659835516 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff25f7fe640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2601036e0 0x7ff260198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff25ffff640 1 -- 172.21.15.31:0/1659835516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2601036e0 msgr2=0x7ff260198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff25ffff640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2601036e0 0x7ff260198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.170+0000 7ff25ffff640 1 -- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2601082a0 msgr2=0x7ff26019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.171+0000 7ff25ffff640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2601082a0 0x7ff26019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.171+0000 7ff25ffff640 1 -- 172.21.15.31:0/1659835516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26019d570 con 0x7ff260102d30 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.171+0000 7ff264f0c640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2601082a0 0x7ff26019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.171+0000 7ff25ffff640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff260102d30 0x7ff2601985c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff250009230 tx=0x7ff250004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.171+0000 7ff25d7fa640 1 -- 172.21.15.31:0/1659835516 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25003f030 con 0x7ff260102d30 2024-09-16T00:42:15.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.171+0000 7ff25d7fa640 1 -- 172.21.15.31:0/1659835516 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff250003ad0 con 0x7ff260102d30 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.171+0000 7ff25d7fa640 1 -- 172.21.15.31:0/1659835516 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250035070 con 0x7ff260102d30 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.171+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26019d800 con 0x7ff260102d30 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.171+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26019dbd0 con 0x7ff260102d30 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.172+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff260103110 con 0x7ff260102d30 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.174+0000 7ff25d7fa640 1 -- 172.21.15.31:0/1659835516 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff250042030 con 0x7ff260102d30 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.174+0000 7ff25d7fa640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2480a32c0 0x7ff2480a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.175+0000 7ff25d7fa640 1 -- 172.21.15.31:0/1659835516 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7ff2500f0140 con 0x7ff260102d30 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.175+0000 7ff25f7fe640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2480a32c0 0x7ff2480a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.175+0000 7ff25f7fe640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2480a32c0 0x7ff2480a5780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2600fee50 tx=0x7ff25400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.178+0000 7ff25d7fa640 1 -- 172.21.15.31:0/1659835516 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2500bc460 con 0x7ff260102d30 2024-09-16T00:42:15.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:15 smithi031 ceph-mon[30956]: pgmap v389: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-16T00:42:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.396+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff260199920 con 0x7ff260102d30 2024-09-16T00:42:15.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.396+0000 7ff25d7fa640 1 -- 172.21.15.31:0/1659835516 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7ff2500c0760 con 0x7ff260102d30 2024-09-16T00:42:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:42:15.397 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:42:15.398 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:42:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.399+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2480a32c0 msgr2=0x7ff2480a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.399+0000 7ff26696f640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2480a32c0 0x7ff2480a5780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2600fee50 tx=0x7ff25400b390 comp rx=0 tx=0).stop 2024-09-16T00:42:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.399+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff260102d30 msgr2=0x7ff2601985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:15.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.399+0000 7ff26696f640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff260102d30 0x7ff2601985c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff250009230 tx=0x7ff250004270 comp rx=0 tx=0).stop 2024-09-16T00:42:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.400+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 shutdown_connections 2024-09-16T00:42:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.400+0000 7ff26696f640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2601082a0 0x7ff26019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.400+0000 7ff26696f640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7ff2480a32c0 0x7ff2480a5780 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.400+0000 7ff26696f640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2601036e0 0x7ff260198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.400+0000 7ff26696f640 1 --2- 172.21.15.31:0/1659835516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff260102d30 0x7ff2601985c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.400+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 >> 172.21.15.31:0/1659835516 conn(0x7ff2600fc7d0 msgr2=0x7ff260105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.400+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 shutdown_connections 2024-09-16T00:42:15.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:15.400+0000 7ff26696f640 1 -- 172.21.15.31:0/1659835516 wait complete. 2024-09-16T00:42:15.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:15 smithi081 ceph-mon[35539]: pgmap v389: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-16T00:42:15.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:15 smithi102 ceph-mon[35364]: pgmap v389: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-16T00:42:16.116 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 923bb43 --release --await' 2024-09-16T00:42:16.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 923bb43 --release --await 2024-09-16T00:42:16.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:16 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1659835516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:16.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:16 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1659835516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:16.509 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:16.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:16 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1659835516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.057+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/2468759562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 msgr2=0x7f0cc81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.057+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/2468759562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 0x7f0cc81029b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009930 tx=0x7f0cbc02a4d0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.057+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/2468759562 shutdown_connections 2024-09-16T00:42:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.057+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/2468759562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc8102ef0 0x7f0cc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.057+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/2468759562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 0x7f0cc81029b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.057+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/2468759562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cc8100e20 0x7f0cc8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.057+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/2468759562 >> 172.21.15.31:0/2468759562 conn(0x7f0cc80fc7d0 msgr2=0x7f0cc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.057+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/2468759562 shutdown_connections 2024-09-16T00:42:17.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.057+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/2468759562 wait complete. 2024-09-16T00:42:17.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.059+0000 7f0ccd9cb640 1 Processor -- start 2024-09-16T00:42:17.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.060+0000 7f0ccd9cb640 1 -- start start 2024-09-16T00:42:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0ccd9cb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cc8100e20 0x7f0cc81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0ccd9cb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 0x7f0cc8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0ccd9cb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc8102ef0 0x7f0cc819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0ccd9cb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc8110f50 con 0x7f0cc8100e20 2024-09-16T00:42:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0ccd9cb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc8110dd0 con 0x7f0cc8102570 2024-09-16T00:42:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0ccd9cb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81110d0 con 0x7f0cc8102ef0 2024-09-16T00:42:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0cc67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 0x7f0cc8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0cc67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 0x7f0cc8198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:51014/0 (socket says 172.21.15.31:51014) 2024-09-16T00:42:17.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0cc67fc640 1 -- 172.21.15.31:0/4151084337 learned_addr learned my addr 172.21.15.31:0/4151084337 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0cc77fe640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc8102ef0 0x7f0cc819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.061+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cc8100e20 0x7f0cc81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0cc67fc640 1 -- 172.21.15.31:0/4151084337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc8102ef0 msgr2=0x7f0cc819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc8102ef0 0x7f0cc819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0cc67fc640 1 -- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cc8100e20 msgr2=0x7f0cc81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cc8100e20 0x7f0cc81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0cc67fc640 1 -- 172.21.15.31:0/4151084337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc819d580 con 0x7f0cc8102570 2024-09-16T00:42:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cc8100e20 0x7f0cc81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:17.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 0x7f0cc8198d20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009230 tx=0x7f0cbc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:17.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0ccc9c9640 1 -- 172.21.15.31:0/4151084337 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc03f030 con 0x7f0cc8102570 2024-09-16T00:42:17.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/4151084337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc819d810 con 0x7f0cc8102570 2024-09-16T00:42:17.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0ccc9c9640 1 -- 172.21.15.31:0/4151084337 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc003ad0 con 0x7f0cc8102570 2024-09-16T00:42:17.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.062+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/4151084337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc819dc80 con 0x7f0cc8102570 2024-09-16T00:42:17.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.063+0000 7f0ccc9c9640 1 -- 172.21.15.31:0/4151084337 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc035070 con 0x7f0cc8102570 2024-09-16T00:42:17.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.066+0000 7f0ccfda4740 1 Processor -- start 2024-09-16T00:42:17.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.066+0000 7f0ccfda4740 1 -- start start 2024-09-16T00:42:17.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.066+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 0x562c982ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562c9823b620 con 0x562c982d6880 2024-09-16T00:42:17.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562c981e2e50 con 0x562c982e9a30 2024-09-16T00:42:17.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562c982375e0 con 0x562c982d5f60 2024-09-16T00:42:17.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0ccc9c9640 1 -- 172.21.15.31:0/4151084337 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0cbc042030 con 0x7f0cc8102570 2024-09-16T00:42:17.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0cc67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 0x562c982ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.067+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47284/0 (socket says 172.21.15.31:47284) 2024-09-16T00:42:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.068+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/3831900374 learned_addr learned my addr 172.21.15.31:0/3831900374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.068+0000 7f0cc77fe640 1 -- 172.21.15.31:0/3831900374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 msgr2=0x562c982ebe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.068+0000 7f0cc77fe640 1 --2- 172.21.15.31:0/3831900374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 0x562c982ebe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.068+0000 7f0cc77fe640 1 -- 172.21.15.31:0/3831900374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 msgr2=0x562c982e94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.068+0000 7f0cc77fe640 1 --2- 172.21.15.31:0/3831900374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.068+0000 7f0cc77fe640 1 -- 172.21.15.31:0/3831900374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c981d3d40 con 0x562c982d5f60 2024-09-16T00:42:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.068+0000 7f0ccc9c9640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0cb00a3340 0x7f0cb00a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.068+0000 7f0ccc9c9640 1 -- 172.21.15.31:0/4151084337 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f0cbc0f0300 con 0x7f0cc8102570 2024-09-16T00:42:17.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.068+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0cb00a3340 0x7f0cb00a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.069+0000 7f0cc77fe640 1 --2- 172.21.15.31:0/3831900374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d6340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009f00 tx=0x7f0cc0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a47b5220a6b80e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:17.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.069+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0cb00a3340 0x7f0cb00a5800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8005d00 tx=0x7f0cb800b660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:17.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.069+0000 7f0cadffb640 1 -- 172.21.15.31:0/3831900374 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0004d20 con 0x562c982d5f60 2024-09-16T00:42:17.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.069+0000 7f0cadffb640 1 -- 172.21.15.31:0/3831900374 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc0003e50 con 0x562c982d5f60 2024-09-16T00:42:17.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.069+0000 7f0cadffb640 1 -- 172.21.15.31:0/3831900374 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0005020 con 0x562c982d5f60 2024-09-16T00:42:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.070+0000 7f0ccfda4740 1 -- 172.21.15.31:0/3831900374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 msgr2=0x562c982d6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.070+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/3831900374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d6340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009f00 tx=0x7f0cc0009500 comp rx=0 tx=0).stop 2024-09-16T00:42:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.070+0000 7f0ccfda4740 1 -- 172.21.15.31:0/3831900374 shutdown_connections 2024-09-16T00:42:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.070+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/3831900374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 0x562c982ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.070+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/3831900374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.070+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/3831900374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d6340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.070+0000 7f0ccfda4740 1 -- 172.21.15.31:0/3831900374 >> 172.21.15.31:0/3831900374 conn(0x562c982d25d0 msgr2=0x562c982d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.071+0000 7f0ccfda4740 1 -- 172.21.15.31:0/3831900374 shutdown_connections 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.071+0000 7f0ccfda4740 1 -- 172.21.15.31:0/3831900374 wait complete. 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.071+0000 7f0ccfda4740 1 -- start start 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.071+0000 7f0ccfda4740 1 Processor -- start 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d5f60 0x562c982e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982e9a30 0x562c982e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562c982ec5a0 con 0x562c982d6880 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562c981c3f10 con 0x562c982d5f60 2024-09-16T00:42:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562c982ec720 con 0x562c982e9a30 2024-09-16T00:42:17.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0ccfda4740 10 client.0 _ll_register_callbacks cb 0x562c981e7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:17.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0cc67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0cc67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47300/0 (socket says 172.21.15.31:47300) 2024-09-16T00:42:17.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0cc67fc640 1 -- 172.21.15.31:0/963623927 learned_addr learned my addr 172.21.15.31:0/963623927 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:17.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.072+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d5f60 0x562c982e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/963623927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982e9a30 msgr2=0x562c982e74d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:42:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982e9a30 0x562c982e74d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/963623927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 msgr2=0x562c982e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/963623927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c97f4a580 con 0x562c982d5f60 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d5f60 0x562c982e6a50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8013fd0 tx=0x7f0cb80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7428fea20e284039 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/963623927 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb801f830 con 0x562c982d5f60 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0ccfda4740 1 -- 172.21.15.31:0/963623927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc0003e50 con 0x562c982d5f60 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.073+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/963623927 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f0cb801f9d0 con 0x562c982d5f60 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/963623927 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0cb801fbd0 con 0x562c982d5f60 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ccfda4740 10 client.25166 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ccfda4740 1 -- 172.21.15.31:0/963623927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562c98014960 con 0x562c982d5f60 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/963623927 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc0003e50 con 0x562c982d5f60 2024-09-16T00:42:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/963623927 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f0cb801d3b0 con 0x562c982d5f60 2024-09-16T00:42:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ccfda4740 10 client.25166 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ccfda4740 10 client.25166 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:42:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ccfda4740 20 client.25166 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562c982f96a0 0x562c982fba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ccfda4740 4 client.25166 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:17.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.074+0000 7f0ccfda4740 1 -- 172.21.15.31:0/963623927 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c98248b90 con 0x562c982f96a0 2024-09-16T00:42:17.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.075+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562c982f96a0 0x562c982fba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.075+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562c982f96a0 0x562c982fba90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fddd8a41bfcd939e server_cookie=4753ae6a70723a0c in_seq=0 out_seq=0 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.075+0000 7f0ca6ffd640 10 client.25166 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.076+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/963623927 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c98248b90 con 0x562c982f96a0 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.076+0000 7f0ca6ffd640 10 client.25166 handle_command_reply: tid=0 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.076+0000 7f0ccfda4740 1 client.25166 shutdown 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.078+0000 7f0ccfda4740 1 -- 172.21.15.31:0/963623927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d5f60 msgr2=0x562c982e6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.078+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d5f60 0x562c982e6a50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8013fd0 tx=0x7f0cb80046c0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.078+0000 7f0ccfda4740 1 -- 172.21.15.31:0/963623927 shutdown_connections 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.078+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562c982f96a0 0x562c982fba90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.078+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982e9a30 0x562c982e74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.078+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d6880 0x562c982e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.078+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/963623927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d5f60 0x562c982e6a50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.078+0000 7f0ccfda4740 1 -- 172.21.15.31:0/963623927 >> 172.21.15.31:0/963623927 conn(0x562c982d25d0 msgr2=0x562c982d8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:17.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.080+0000 7f0ca6ffd640 0 client.25166 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:17.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.080+0000 7f0ccfda4740 1 -- 172.21.15.31:0/963623927 shutdown_connections 2024-09-16T00:42:17.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.080+0000 7f0ccfda4740 1 -- 172.21.15.31:0/963623927 wait complete. 2024-09-16T00:42:17.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.080+0000 7f0ccfda4740 20 client.25166 trim_cache size 0 max 16384 2024-09-16T00:42:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.112+0000 7f0ccfda4740 1 Processor -- start 2024-09-16T00:42:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.112+0000 7f0ccfda4740 1 -- start start 2024-09-16T00:42:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.112+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.112+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d6880 0x562c98301360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.112+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982e9a30 0x562c983018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.112+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81aaa90 con 0x562c982e9a30 2024-09-16T00:42:17.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.112+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81ad310 con 0x562c982d6880 2024-09-16T00:42:17.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.112+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81ab100 con 0x562c982d5f60 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.112+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982e9a30 0x562c983018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d6880 0x562c98301360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:58012/0 (socket says 172.21.15.31:58012) 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982e9a30 0x562c983018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47306/0 (socket says 172.21.15.31:47306) 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/915947620 learned_addr learned my addr 172.21.15.31:0/915947620 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/915947620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d6880 msgr2=0x562c98301360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/915947620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d6880 0x562c98301360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/915947620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982e9a30 msgr2=0x562c983018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/915947620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982e9a30 0x562c983018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/915947620 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c982d7470 con 0x562c982d5f60 2024-09-16T00:42:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/915947620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d6880 0x562c98301360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/915947620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d1580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8017040 tx=0x7f0cb800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca25534b887d3e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.113+0000 7f0cad7fa640 1 -- 172.21.15.31:0/915947620 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb803b070 con 0x562c982d5f60 2024-09-16T00:42:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.114+0000 7f0cad7fa640 1 -- 172.21.15.31:0/915947620 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb801d3b0 con 0x562c982d5f60 2024-09-16T00:42:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.114+0000 7f0cad7fa640 1 -- 172.21.15.31:0/915947620 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8035700 con 0x562c982d5f60 2024-09-16T00:42:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.114+0000 7f0ccfda4740 1 -- 172.21.15.31:0/915947620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 msgr2=0x562c982d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/915947620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d1580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8017040 tx=0x7f0cb800bc80 comp rx=0 tx=0).stop 2024-09-16T00:42:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 -- 172.21.15.31:0/915947620 shutdown_connections 2024-09-16T00:42:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/915947620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982e9a30 0x562c983018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/915947620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982d6880 0x562c98301360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/915947620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d5f60 0x562c982d1580 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 -- 172.21.15.31:0/915947620 >> 172.21.15.31:0/915947620 conn(0x562c982d25d0 msgr2=0x562c982e7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 -- 172.21.15.31:0/915947620 shutdown_connections 2024-09-16T00:42:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 -- 172.21.15.31:0/915947620 wait complete. 2024-09-16T00:42:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 -- start start 2024-09-16T00:42:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.115+0000 7f0ccfda4740 1 Processor -- start 2024-09-16T00:42:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d5f60 0x562c982f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d6880 0x562c982ee980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0ccfda4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 0x562c982eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81ab7d0 con 0x562c982d5f60 2024-09-16T00:42:17.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81acc40 con 0x562c982e9a30 2024-09-16T00:42:17.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0ccfda4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81abea0 con 0x562c982d6880 2024-09-16T00:42:17.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0ccfda4740 10 client.0 _ll_register_callbacks cb 0x7f0cb8004110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:17.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d5f60 0x562c982f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d5f60 0x562c982f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47310/0 (socket says 172.21.15.31:47310) 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/2174677702 learned_addr learned my addr 172.21.15.31:0/2174677702 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0cc77fe640 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 0x562c982eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.116+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d6880 0x562c982ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/2174677702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d6880 msgr2=0x562c982ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d6880 0x562c982ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/2174677702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 msgr2=0x562c982eeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 0x562c982eeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0cc6ffd640 1 -- 172.21.15.31:0/2174677702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c982ef6c0 con 0x562c982d5f60 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0cc6ffd640 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d5f60 0x562c982f20c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8013fd0 tx=0x7f0cb800b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17198c94c08be7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:42:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/2174677702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8044030 con 0x562c982d5f60 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/2174677702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f0cb801f840 con 0x562c982d5f60 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.117+0000 7f0ccfda4740 1 -- 172.21.15.31:0/2174677702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb801d3b0 con 0x562c982d5f60 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.118+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/2174677702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0cb804f050 con 0x562c982d5f60 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.118+0000 7f0ccfda4740 10 client.15468 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.118+0000 7f0ccfda4740 1 -- 172.21.15.31:0/2174677702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562c98365320 con 0x562c982d5f60 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.118+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/2174677702 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb801d3b0 con 0x562c982d5f60 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.118+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/2174677702 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f0cb8040560 con 0x562c982d5f60 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.119+0000 7f0ccfda4740 10 client.15468 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.119+0000 7f0ccfda4740 10 client.15468 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.119+0000 7f0ccfda4740 20 client.15468 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.119+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562c983659d0 0x562c98367df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.119+0000 7f0ccfda4740 4 client.15468 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "923bb43", "release": true, "await": true}] 2024-09-16T00:42:17.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.119+0000 7f0ccfda4740 1 -- 172.21.15.31:0/2174677702 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "set_id": "923bb43", "release": true, "await": true}) -- 0x562c982430a0 con 0x562c983659d0 2024-09-16T00:42:17.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.119+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562c983659d0 0x562c98367df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:17.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.119+0000 7f0ca6ffd640 10 client.15468 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:17.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.119+0000 7f0cc67fc640 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562c983659d0 0x562c98367df0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a131d91f1d3825 server_cookie=e4f2cb866c58655c in_seq=0 out_seq=0 2024-09-16T00:42:17.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ca6ffd640 1 -- 172.21.15.31:0/2174677702 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x562c982430a0 con 0x562c983659d0 2024-09-16T00:42:17.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ca6ffd640 10 client.15468 handle_command_reply: tid=0 2024-09-16T00:42:17.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ccfda4740 1 client.15468 shutdown 2024-09-16T00:42:17.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ccfda4740 1 -- 172.21.15.31:0/2174677702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d5f60 msgr2=0x562c982f20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:17.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d5f60 0x562c982f20c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8013fd0 tx=0x7f0cb800b920 comp rx=0 tx=0).stop 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ccfda4740 1 -- 172.21.15.31:0/2174677702 shutdown_connections 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c982e9a30 0x562c982eeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x562c983659d0 0x562c98367df0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ca6ffd640 0 client.15468 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c982d6880 0x562c982ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ccfda4740 1 --2- 172.21.15.31:0/2174677702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562c982d5f60 0x562c982f20c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.121+0000 7f0ccfda4740 1 -- 172.21.15.31:0/2174677702 >> 172.21.15.31:0/2174677702 conn(0x562c982d25d0 msgr2=0x562c982d7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.122+0000 7f0ccfda4740 1 -- 172.21.15.31:0/2174677702 shutdown_connections 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.122+0000 7f0ccfda4740 1 -- 172.21.15.31:0/2174677702 wait complete. 2024-09-16T00:42:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.122+0000 7f0ccfda4740 20 client.15468 trim_cache size 0 max 16384 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 4, 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "923bb43": { 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 4, 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:42:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 150.2, 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:42:17.126 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:42:17.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0cb00a3340 msgr2=0x7f0cb00a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:17.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0cb00a3340 0x7f0cb00a5800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8005d00 tx=0x7f0cb800b660 comp rx=0 tx=0).stop 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/4151084337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 msgr2=0x7f0cc8198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 0x7f0cc8198d20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009230 tx=0x7f0cbc004270 comp rx=0 tx=0).stop 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/4151084337 shutdown_connections 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0cb00a3340 0x7f0cb00a5800 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cc8102ef0 0x7f0cc819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cc8102570 0x7f0cc8198d20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 --2- 172.21.15.31:0/4151084337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cc8100e20 0x7f0cc81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.124+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/4151084337 >> 172.21.15.31:0/4151084337 conn(0x7f0cc80fc7d0 msgr2=0x7f0cc8069fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.125+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/4151084337 shutdown_connections 2024-09-16T00:42:17.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:17.125+0000 7f0ccd9cb640 1 -- 172.21.15.31:0/4151084337 wait complete. 2024-09-16T00:42:17.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:17 smithi081 ceph-mon[35539]: pgmap v390: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-16T00:42:17.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:17 smithi102 ceph-mon[35364]: pgmap v390: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-16T00:42:17.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:17 smithi031 ceph-mon[30956]: pgmap v390: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-16T00:42:17.952 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '923bb43', total seconds elapsed: 33.8 2024-09-16T00:42:19.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:19 smithi081 ceph-mon[35539]: pgmap v391: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-16T00:42:19.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:19 smithi102 ceph-mon[35364]: pgmap v391: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-16T00:42:19.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:19 smithi031 ceph-mon[30956]: pgmap v391: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-16T00:42:21.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:21 smithi081 ceph-mon[35539]: pgmap v392: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-16T00:42:21.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:21 smithi102 ceph-mon[35364]: pgmap v392: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-16T00:42:21.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:21 smithi031 ceph-mon[30956]: pgmap v392: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-16T00:42:23.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:23 smithi081 ceph-mon[35539]: pgmap v393: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:42:23.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:23 smithi102 ceph-mon[35364]: pgmap v393: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:42:23.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:23 smithi031 ceph-mon[30956]: pgmap v393: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:42:25.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:25 smithi081 ceph-mon[35539]: pgmap v394: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:42:25.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:25 smithi102 ceph-mon[35364]: pgmap v394: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:42:25.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:25 smithi031 ceph-mon[30956]: pgmap v394: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:42:25.682 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 94% (12166/12889) Updating files: 95% (12245/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-16T00:42:26.537 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T00:42:26.537 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:26.537 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T00:42:26.537 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T00:42:26.537 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T00:42:26.537 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:26.538 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T00:42:26.538 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T00:42:26.538 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:26.538 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2024-09-16T00:42:26.538 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:26.538 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2024-09-16T00:42:26.538 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:26.538 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2024-09-16T00:42:26.538 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:26.539 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T00:42:26.539 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:26.539 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T00:42:26.542 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T00:42:26.569 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T00:42:26.571 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:42:26.571 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T00:42:27.365 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T00:42:27.440 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T00:42:27.492 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:42:27.492 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:42:27.492 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T00:42:27.499 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:27 smithi081 ceph-mon[35539]: pgmap v395: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:42:27.521 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:42:27.529 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:42:27.529 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T00:42:27.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:27 smithi102 ceph-mon[35364]: pgmap v395: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:42:27.589 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-16T00:42:27.590 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-16T00:42:27.591 DEBUG:teuthology.orchestra.run.smithi031:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-16T00:42:27.615 INFO:tasks.workunit.client.0.smithi031.stdout:getting blogbench 2024-09-16T00:42:27.615 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 'getting blogbench' 2024-09-16T00:42:27.615 INFO:tasks.workunit.client.0.smithi031.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-16T00:42:27.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:27 smithi031 ceph-mon[30956]: pgmap v395: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:42:27.631 INFO:tasks.workunit.client.0.smithi031.stderr:--2024-09-16 00:42:27-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-16T00:42:27.632 INFO:tasks.workunit.client.0.smithi031.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-16T00:42:27.653 INFO:tasks.workunit.client.0.smithi031.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-16T00:42:27.673 INFO:tasks.workunit.client.0.smithi031.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-16T00:42:27.673 INFO:tasks.workunit.client.0.smithi031.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-16T00:42:27.675 INFO:tasks.workunit.client.0.smithi031.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-16T00:42:27.677 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:27.715 INFO:tasks.workunit.client.0.smithi031.stderr: 0K .......... .......... .......... .......... .......... 49% 1.24M 0s 2024-09-16T00:42:27.735 INFO:tasks.workunit.client.0.smithi031.stderr: 50K .......... .......... .......... .......... .......... 99% 2.43M 0s 2024-09-16T00:42:27.735 INFO:tasks.workunit.client.0.smithi031.stderr: 100K 100% 1.74T=0.06s 2024-09-16T00:42:27.735 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:27.735 INFO:tasks.workunit.client.0.smithi031.stderr:2024-09-16 00:42:27 (1.65 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-16T00:42:27.735 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:42:27.735 INFO:tasks.workunit.client.0.smithi031.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-16T00:42:27.766 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/ 2024-09-16T00:42:27.767 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/m4/ 2024-09-16T00:42:27.768 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-16T00:42:27.769 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/man/ 2024-09-16T00:42:27.770 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/man/blogbench.8 2024-09-16T00:42:27.771 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/man/Makefile.am 2024-09-16T00:42:27.771 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/man/Makefile.in 2024-09-16T00:42:27.772 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/ 2024-09-16T00:42:27.773 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-16T00:42:27.774 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-16T00:42:27.776 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/blogbench.c 2024-09-16T00:42:27.776 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/blogbench.h 2024-09-16T00:42:27.777 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-16T00:42:27.779 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-16T00:42:27.780 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-16T00:42:27.782 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-16T00:42:27.783 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-16T00:42:27.784 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/Makefile.am 2024-09-16T00:42:27.786 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/Makefile.in 2024-09-16T00:42:27.787 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/process.c 2024-09-16T00:42:27.789 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/globals.h 2024-09-16T00:42:27.790 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/commenter.c 2024-09-16T00:42:27.791 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/rewriter.c 2024-09-16T00:42:27.792 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/reader.c 2024-09-16T00:42:27.794 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/writer.c 2024-09-16T00:42:27.795 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/src/helpers.c 2024-09-16T00:42:27.797 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/NEWS 2024-09-16T00:42:27.798 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/compile 2024-09-16T00:42:27.799 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/depcomp 2024-09-16T00:42:27.800 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/aclocal.m4 2024-09-16T00:42:27.802 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/README 2024-09-16T00:42:27.803 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/configure 2024-09-16T00:42:27.807 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/configure.ac 2024-09-16T00:42:27.808 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/config.guess 2024-09-16T00:42:27.809 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/install-sh 2024-09-16T00:42:27.811 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/config.sub 2024-09-16T00:42:27.812 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/missing 2024-09-16T00:42:27.813 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/mkinstalldirs 2024-09-16T00:42:27.815 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/Makefile.am 2024-09-16T00:42:27.816 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/Makefile.in 2024-09-16T00:42:27.817 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/config.h.in 2024-09-16T00:42:27.818 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/AUTHORS 2024-09-16T00:42:27.819 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/INSTALL 2024-09-16T00:42:27.820 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/ChangeLog 2024-09-16T00:42:27.821 INFO:tasks.workunit.client.0.smithi031.stdout:blogbench-1.0/COPYING 2024-09-16T00:42:27.823 INFO:tasks.workunit.client.0.smithi031.stderr:+ cd blogbench-1.0/ 2024-09-16T00:42:27.823 INFO:tasks.workunit.client.0.smithi031.stdout:making blogbench 2024-09-16T00:42:27.823 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 'making blogbench' 2024-09-16T00:42:27.823 INFO:tasks.workunit.client.0.smithi031.stderr:+ ./configure 2024-09-16T00:42:27.924 INFO:tasks.workunit.client.0.smithi031.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-16T00:42:28.931 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether build environment is sane... yes 2024-09-16T00:42:28.941 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gawk... gawk 2024-09-16T00:42:28.957 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether make sets $(MAKE)... yes 2024-09-16T00:42:28.966 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc... gcc 2024-09-16T00:42:29.022 INFO:tasks.workunit.client.0.smithi031.stdout:checking for C compiler default output file name... a.out 2024-09-16T00:42:29.024 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether the C compiler works... yes 2024-09-16T00:42:29.026 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether we are cross compiling... no 2024-09-16T00:42:29.065 INFO:tasks.workunit.client.0.smithi031.stdout:checking for suffix of executables... 2024-09-16T00:42:29.101 INFO:tasks.workunit.client.0.smithi031.stdout:checking for suffix of object files... o 2024-09-16T00:42:29.134 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether we are using the GNU C compiler... yes 2024-09-16T00:42:29.167 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether gcc accepts -g... yes 2024-09-16T00:42:29.228 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-16T00:42:29.282 INFO:tasks.workunit.client.0.smithi031.stdout:checking for style of include used by make... GNU 2024-09-16T00:42:29.383 INFO:tasks.workunit.client.0.smithi031.stdout:checking dependency style of gcc... gcc3 2024-09-16T00:42:29.384 INFO:tasks.workunit.client.0.smithi031.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-16T00:42:29.433 INFO:tasks.workunit.client.0.smithi031.stdout:checking how to run the C preprocessor... gcc -E 2024-09-16T00:42:29.474 INFO:tasks.workunit.client.0.smithi031.stdout:checking for egrep... grep -E 2024-09-16T00:42:29.483 INFO:tasks.workunit.client.0.smithi031.stdout:checking for AIX... no 2024-09-16T00:42:29.496 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:29 smithi081 ceph-mon[35539]: pgmap v396: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:42:29.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:29 smithi102 ceph-mon[35364]: pgmap v396: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:42:29.551 INFO:tasks.workunit.client.0.smithi031.stdout:checking for library containing strerror... none required 2024-09-16T00:42:29.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:29 smithi031 ceph-mon[30956]: pgmap v396: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:42:29.735 INFO:tasks.workunit.client.0.smithi031.stdout:checking for ANSI C header files... yes 2024-09-16T00:42:29.771 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-16T00:42:29.820 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/types.h... yes 2024-09-16T00:42:29.866 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/stat.h... yes 2024-09-16T00:42:29.912 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stdlib.h... yes 2024-09-16T00:42:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:checking for string.h... yes 2024-09-16T00:42:30.008 INFO:tasks.workunit.client.0.smithi031.stdout:checking for memory.h... yes 2024-09-16T00:42:30.055 INFO:tasks.workunit.client.0.smithi031.stdout:checking for strings.h... yes 2024-09-16T00:42:30.104 INFO:tasks.workunit.client.0.smithi031.stdout:checking for inttypes.h... yes 2024-09-16T00:42:30.154 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stdint.h... yes 2024-09-16T00:42:30.207 INFO:tasks.workunit.client.0.smithi031.stdout:checking for unistd.h... yes 2024-09-16T00:42:30.218 INFO:tasks.workunit.client.0.smithi031.stdout:checking for memory.h... (cached) yes 2024-09-16T00:42:30.224 INFO:tasks.workunit.client.0.smithi031.stdout:checking for strings.h... (cached) yes 2024-09-16T00:42:30.228 INFO:tasks.workunit.client.0.smithi031.stdout:checking for unistd.h... (cached) yes 2024-09-16T00:42:30.273 INFO:tasks.workunit.client.0.smithi031.stdout:checking locale.h usability... yes 2024-09-16T00:42:30.293 INFO:tasks.workunit.client.0.smithi031.stdout:checking locale.h presence... yes 2024-09-16T00:42:30.294 INFO:tasks.workunit.client.0.smithi031.stdout:checking for locale.h... yes 2024-09-16T00:42:30.312 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stdlib.h... (cached) yes 2024-09-16T00:42:30.317 INFO:tasks.workunit.client.0.smithi031.stdout:checking for string.h... (cached) yes 2024-09-16T00:42:30.380 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/socket.h usability... yes 2024-09-16T00:42:30.401 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/socket.h presence... yes 2024-09-16T00:42:30.402 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/socket.h... yes 2024-09-16T00:42:30.453 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/time.h usability... yes 2024-09-16T00:42:30.473 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/time.h presence... yes 2024-09-16T00:42:30.473 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/time.h... yes 2024-09-16T00:42:30.525 INFO:tasks.workunit.client.0.smithi031.stdout:checking fcntl.h usability... yes 2024-09-16T00:42:30.545 INFO:tasks.workunit.client.0.smithi031.stdout:checking fcntl.h presence... yes 2024-09-16T00:42:30.546 INFO:tasks.workunit.client.0.smithi031.stdout:checking for fcntl.h... yes 2024-09-16T00:42:30.602 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/fcntl.h usability... yes 2024-09-16T00:42:30.623 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/fcntl.h presence... yes 2024-09-16T00:42:30.623 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/fcntl.h... yes 2024-09-16T00:42:30.725 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/param.h usability... yes 2024-09-16T00:42:30.748 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/param.h presence... yes 2024-09-16T00:42:30.748 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/param.h... yes 2024-09-16T00:42:30.799 INFO:tasks.workunit.client.0.smithi031.stdout:checking ioctl.h usability... no 2024-09-16T00:42:30.819 INFO:tasks.workunit.client.0.smithi031.stdout:checking ioctl.h presence... no 2024-09-16T00:42:30.820 INFO:tasks.workunit.client.0.smithi031.stdout:checking for ioctl.h... no 2024-09-16T00:42:30.873 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/ioctl.h usability... yes 2024-09-16T00:42:30.893 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/ioctl.h presence... yes 2024-09-16T00:42:30.893 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/ioctl.h... yes 2024-09-16T00:42:30.947 INFO:tasks.workunit.client.0.smithi031.stdout:checking return type of signal handlers... void 2024-09-16T00:42:30.991 INFO:tasks.workunit.client.0.smithi031.stdout:checking for size_t... yes 2024-09-16T00:42:31.028 INFO:tasks.workunit.client.0.smithi031.stdout:checking for dev_t... yes 2024-09-16T00:42:31.072 INFO:tasks.workunit.client.0.smithi031.stdout:checking for short... yes 2024-09-16T00:42:31.147 INFO:tasks.workunit.client.0.smithi031.stdout:checking size of short... 2 2024-09-16T00:42:31.190 INFO:tasks.workunit.client.0.smithi031.stdout:checking for int... yes 2024-09-16T00:42:31.258 INFO:tasks.workunit.client.0.smithi031.stdout:checking size of int... 4 2024-09-16T00:42:31.301 INFO:tasks.workunit.client.0.smithi031.stdout:checking for long... yes 2024-09-16T00:42:31.368 INFO:tasks.workunit.client.0.smithi031.stdout:checking size of long... 8 2024-09-16T00:42:31.412 INFO:tasks.workunit.client.0.smithi031.stdout:checking for long long... yes 2024-09-16T00:42:31.481 INFO:tasks.workunit.client.0.smithi031.stdout:checking size of long long... 8 2024-09-16T00:42:31.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:31 smithi081 ceph-mon[35539]: pgmap v397: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 1 op/s 2024-09-16T00:42:31.524 INFO:tasks.workunit.client.0.smithi031.stdout:checking for ssize_t... yes 2024-09-16T00:42:31.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:31 smithi102 ceph-mon[35364]: pgmap v397: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 1 op/s 2024-09-16T00:42:31.563 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sig_atomic_t... yes 2024-09-16T00:42:31.611 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether gcc needs -traditional... no 2024-09-16T00:42:31.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:31 smithi031 ceph-mon[30956]: pgmap v397: 129 pgs: 129 active+clean; 656 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 1 op/s 2024-09-16T00:42:31.645 INFO:tasks.workunit.client.0.smithi031.stdout:checking for an ANSI C-conforming const... yes 2024-09-16T00:42:31.668 INFO:tasks.workunit.client.0.smithi031.stdout:checking for inline... inline 2024-09-16T00:42:31.727 INFO:tasks.workunit.client.0.smithi031.stdout:checking for working alloca.h... yes 2024-09-16T00:42:31.774 INFO:tasks.workunit.client.0.smithi031.stdout:checking for alloca... yes 2024-09-16T00:42:31.840 INFO:tasks.workunit.client.0.smithi031.stdout:checking for working memcmp... yes 2024-09-16T00:42:31.896 INFO:tasks.workunit.client.0.smithi031.stdout:checking for strftime... yes 2024-09-16T00:42:31.969 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-16T00:42:32.030 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether stat accepts an empty string... no 2024-09-16T00:42:32.085 INFO:tasks.workunit.client.0.smithi031.stdout:checking for vprintf... yes 2024-09-16T00:42:32.156 INFO:tasks.workunit.client.0.smithi031.stdout:checking for _doprnt... no 2024-09-16T00:42:32.178 INFO:tasks.workunit.client.0.smithi031.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-16T00:42:32.190 INFO:tasks.workunit.client.0.smithi031.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-16T00:42:32.252 INFO:tasks.workunit.client.0.smithi031.stdout:checking for the pthreads library -lpthreads... no 2024-09-16T00:42:32.306 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether pthreads work without any flags... yes 2024-09-16T00:42:32.352 INFO:tasks.workunit.client.0.smithi031.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-16T00:42:32.352 INFO:tasks.workunit.client.0.smithi031.stdout:checking if more special flags are required for pthreads... no 2024-09-16T00:42:32.353 INFO:tasks.workunit.client.0.smithi031.stdout:checking for cc_r... gcc 2024-09-16T00:42:32.428 INFO:tasks.workunit.client.0.smithi031.stdout:checking for setrlimit... yes 2024-09-16T00:42:32.488 INFO:tasks.workunit.client.0.smithi031.stdout:checking for setproctitle... no 2024-09-16T00:42:32.546 INFO:tasks.workunit.client.0.smithi031.stdout:checking for getopt_long... yes 2024-09-16T00:42:32.605 INFO:tasks.workunit.client.0.smithi031.stdout:checking for strtoull... yes 2024-09-16T00:42:32.663 INFO:tasks.workunit.client.0.smithi031.stdout:checking for strtoq... yes 2024-09-16T00:42:32.724 INFO:tasks.workunit.client.0.smithi031.stdout:checking for snprintf... yes 2024-09-16T00:42:32.837 INFO:tasks.workunit.client.0.smithi031.stdout:checking for vsnprintf... yes 2024-09-16T00:42:32.905 INFO:tasks.workunit.client.0.smithi031.stdout:checking for vfprintf... yes 2024-09-16T00:42:32.965 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gethostname... yes 2024-09-16T00:42:33.024 INFO:tasks.workunit.client.0.smithi031.stdout:checking for setlocale... yes 2024-09-16T00:42:33.084 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sync... yes 2024-09-16T00:42:33.191 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether snprintf is C99 conformant... done 2024-09-16T00:42:33.206 INFO:tasks.workunit.client.0.smithi031.stdout:configure: creating ./config.status 2024-09-16T00:42:33.365 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: creating Makefile 2024-09-16T00:42:33.382 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: creating src/Makefile 2024-09-16T00:42:33.401 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: creating man/Makefile 2024-09-16T00:42:33.413 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: creating config.h 2024-09-16T00:42:33.463 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: executing depfiles commands 2024-09-16T00:42:33.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:33 smithi081 ceph-mon[35539]: pgmap v398: 129 pgs: 129 active+clean; 1.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 79 KiB/s wr, 5 op/s 2024-09-16T00:42:33.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:33 smithi102 ceph-mon[35364]: pgmap v398: 129 pgs: 129 active+clean; 1.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 79 KiB/s wr, 5 op/s 2024-09-16T00:42:33.588 INFO:tasks.workunit.client.0.smithi031.stderr:+ make 2024-09-16T00:42:33.597 INFO:tasks.workunit.client.0.smithi031.stdout:make all-recursive 2024-09-16T00:42:33.600 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-16T00:42:33.602 INFO:tasks.workunit.client.0.smithi031.stdout:Making all in src 2024-09-16T00:42:33.609 INFO:tasks.workunit.client.0.smithi031.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-16T00:42:33.609 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-16T00:42:33.609 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-16T00:42:33.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:33 smithi031 ceph-mon[30956]: pgmap v398: 129 pgs: 129 active+clean; 1.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 79 KiB/s wr, 5 op/s 2024-09-16T00:42:33.686 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-16T00:42:33.686 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-16T00:42:33.755 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-16T00:42:33.755 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-16T00:42:33.816 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-16T00:42:33.816 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-16T00:42:33.869 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-16T00:42:33.870 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-16T00:42:33.923 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-16T00:42:33.923 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-16T00:42:33.974 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-16T00:42:33.974 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-16T00:42:34.051 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-16T00:42:34.051 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-16T00:42:34.085 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-16T00:42:34.085 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-16T00:42:34.119 INFO:tasks.workunit.client.0.smithi031.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-16T00:42:34.119 INFO:tasks.workunit.client.0.smithi031.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-16T00:42:34.158 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-16T00:42:34.177 INFO:tasks.workunit.client.0.smithi031.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-16T00:42:34.177 INFO:tasks.workunit.client.0.smithi031.stdout:Making all in man 2024-09-16T00:42:34.200 INFO:tasks.workunit.client.0.smithi031.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-16T00:42:34.200 INFO:tasks.workunit.client.0.smithi031.stdout:make[2]: Nothing to be done for 'all'. 2024-09-16T00:42:34.200 INFO:tasks.workunit.client.0.smithi031.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-16T00:42:34.201 INFO:tasks.workunit.client.0.smithi031.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-16T00:42:34.203 INFO:tasks.workunit.client.0.smithi031.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-16T00:42:34.203 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-16T00:42:34.203 INFO:tasks.workunit.client.0.smithi031.stderr:+ cd src 2024-09-16T00:42:34.203 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir blogtest_in 2024-09-16T00:42:34.205 INFO:tasks.workunit.client.0.smithi031.stdout:running blogbench 2024-09-16T00:42:34.206 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 'running blogbench' 2024-09-16T00:42:34.206 INFO:tasks.workunit.client.0.smithi031.stderr:+ ./blogbench -d blogtest_in 2024-09-16T00:42:34.206 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:42:34.206 INFO:tasks.workunit.client.0.smithi031.stdout:Frequency = 10 secs 2024-09-16T00:42:34.206 INFO:tasks.workunit.client.0.smithi031.stdout:Scratch dir = [blogtest_in] 2024-09-16T00:42:35.500 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:35 smithi081 ceph-mon[35539]: pgmap v399: 129 pgs: 129 active+clean; 1.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 89 KiB/s wr, 7 op/s 2024-09-16T00:42:35.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:35 smithi102 ceph-mon[35364]: pgmap v399: 129 pgs: 129 active+clean; 1.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 89 KiB/s wr, 7 op/s 2024-09-16T00:42:35.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:35 smithi031 ceph-mon[30956]: pgmap v399: 129 pgs: 129 active+clean; 1.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 89 KiB/s wr, 7 op/s 2024-09-16T00:42:36.183 INFO:tasks.workunit.client.0.smithi031.stdout:Spawning 3 writers... 2024-09-16T00:42:36.183 INFO:tasks.workunit.client.0.smithi031.stdout:Spawning 1 rewriters... 2024-09-16T00:42:36.183 INFO:tasks.workunit.client.0.smithi031.stdout:Spawning 5 commenters... 2024-09-16T00:42:36.183 INFO:tasks.workunit.client.0.smithi031.stdout:Spawning 100 readers... 2024-09-16T00:42:36.186 INFO:tasks.workunit.client.0.smithi031.stdout:Benchmarking for 30 iterations. 2024-09-16T00:42:36.186 INFO:tasks.workunit.client.0.smithi031.stdout:The test will run during 5 minutes. 2024-09-16T00:42:36.186 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:42:36.186 INFO:tasks.workunit.client.0.smithi031.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-16T00:42:37.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:37 smithi081 ceph-mon[35539]: pgmap v400: 129 pgs: 129 active+clean; 3.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 301 KiB/s wr, 24 op/s 2024-09-16T00:42:37.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:37 smithi102 ceph-mon[35364]: pgmap v400: 129 pgs: 129 active+clean; 3.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 301 KiB/s wr, 24 op/s 2024-09-16T00:42:37.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:37 smithi031 ceph-mon[30956]: pgmap v400: 129 pgs: 129 active+clean; 3.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 301 KiB/s wr, 24 op/s 2024-09-16T00:42:37.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:42:37.909 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:38.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.390+0000 7f70c6f28640 1 -- 172.21.15.31:0/2382191219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 msgr2=0x7f70c010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:38.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.390+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2382191219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 0x7f70c010ec30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009cd0 tx=0x7f70b802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.390+0000 7f70c6f28640 1 -- 172.21.15.31:0/2382191219 shutdown_connections 2024-09-16T00:42:38.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.390+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2382191219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 0x7f70c010ec30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.390+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2382191219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70c0103740 0x7f70c0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.390+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2382191219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70c0102d90 0x7f70c0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.390+0000 7f70c6f28640 1 -- 172.21.15.31:0/2382191219 >> 172.21.15.31:0/2382191219 conn(0x7f70c00fc7d0 msgr2=0x7f70c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.390+0000 7f70c6f28640 1 -- 172.21.15.31:0/2382191219 shutdown_connections 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.390+0000 7f70c6f28640 1 -- 172.21.15.31:0/2382191219 wait complete. 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c6f28640 1 Processor -- start 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c6f28640 1 -- start start 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c6f28640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70c0102d90 0x7f70c01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c6f28640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70c0103740 0x7f70c0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c6f28640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 0x7f70c019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c6f28640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01120d0 con 0x7f70c0108290 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c6f28640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0111f50 con 0x7f70c0103740 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c6f28640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0112250 con 0x7f70c0102d90 2024-09-16T00:42:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c54c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 0x7f70c019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70c0103740 0x7f70c0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c54c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 0x7f70c019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36540/0 (socket says 172.21.15.31:36540) 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c54c5640 1 -- 172.21.15.31:0/2113552228 learned_addr learned my addr 172.21.15.31:0/2113552228 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70c0102d90 0x7f70c01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70bffff640 1 -- 172.21.15.31:0/2113552228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70c0102d90 msgr2=0x7f70c01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70bffff640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70c0102d90 0x7f70c01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70bffff640 1 -- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 msgr2=0x7f70c019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70bffff640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 0x7f70c019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70bffff640 1 -- 172.21.15.31:0/2113552228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c019d560 con 0x7f70c0103740 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.391+0000 7f70c54c5640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 0x7f70c019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.392+0000 7f70bffff640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70c0103740 0x7f70c0198af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e9c0 tx=0x7f70ac00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.392+0000 7f70bdffb640 1 -- 172.21.15.31:0/2113552228 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac00ccc0 con 0x7f70c0103740 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.392+0000 7f70c6f28640 1 -- 172.21.15.31:0/2113552228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c019d850 con 0x7f70c0103740 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.392+0000 7f70bdffb640 1 -- 172.21.15.31:0/2113552228 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70ac010910 con 0x7f70c0103740 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.392+0000 7f70bdffb640 1 -- 172.21.15.31:0/2113552228 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0188e0 con 0x7f70c0103740 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.392+0000 7f70c6f28640 1 -- 172.21.15.31:0/2113552228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c01a5130 con 0x7f70c0103740 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70bdffb640 1 -- 172.21.15.31:0/2113552228 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f70ac010ac0 con 0x7f70c0103740 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c9301740 1 Processor -- start 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c9301740 1 -- start start 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c9301740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x56173912d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c9301740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x561739138140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c9301740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x56173913aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c9301740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5617390932e0 con 0x561739138680 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c9301740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56173903ace0 con 0x56173912d4a0 2024-09-16T00:42:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c9301740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56173908f2a0 con 0x56173912ddc0 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c54c5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x56173912d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x56173913aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c54c5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x56173912d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:35246/0 (socket says 172.21.15.31:35246) 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c54c5640 1 -- 172.21.15.31:0/2008078295 learned_addr learned my addr 172.21.15.31:0/2008078295 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2008078295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x561739138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c54c5640 1 -- 172.21.15.31:0/2008078295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 msgr2=0x561739138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c54c5640 1 --2- 172.21.15.31:0/2008078295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x561739138140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c54c5640 1 -- 172.21.15.31:0/2008078295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 msgr2=0x56173913aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.395+0000 7f70c54c5640 1 --2- 172.21.15.31:0/2008078295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x56173913aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.396+0000 7f70c54c5640 1 -- 172.21.15.31:0/2008078295 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173901f440 con 0x56173912d4a0 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.396+0000 7f70bdffb640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f70b00a3240 0x7f70b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.396+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2008078295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x561739138140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.396+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f70b00a3240 0x7f70b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.396+0000 7f70c54c5640 1 --2- 172.21.15.31:0/2008078295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x56173912d880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70b8031940 tx=0x7f70b8009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2e68df2f150ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.396+0000 7f70bdffb640 1 -- 172.21.15.31:0/2113552228 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f70ac0cff50 con 0x7f70c0103740 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.396+0000 7f70aaffd640 1 -- 172.21.15.31:0/2008078295 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8048070 con 0x56173912d4a0 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.396+0000 7f70aaffd640 1 -- 172.21.15.31:0/2008078295 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b8036070 con 0x56173912d4a0 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.396+0000 7f70aaffd640 1 -- 172.21.15.31:0/2008078295 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b80305a0 con 0x56173912d4a0 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 -- 172.21.15.31:0/2008078295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 msgr2=0x56173912d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f70b00a3240 0x7f70b00a5700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x56173912a890 tx=0x7f70b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 --2- 172.21.15.31:0/2008078295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x56173912d880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70b8031940 tx=0x7f70b8009a20 comp rx=0 tx=0).stop 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 -- 172.21.15.31:0/2008078295 shutdown_connections 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 --2- 172.21.15.31:0/2008078295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x56173913aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 --2- 172.21.15.31:0/2008078295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x561739138140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 --2- 172.21.15.31:0/2008078295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x56173912d880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 -- 172.21.15.31:0/2008078295 >> 172.21.15.31:0/2008078295 conn(0x561739129a30 msgr2=0x561739129e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 -- 172.21.15.31:0/2008078295 shutdown_connections 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 -- 172.21.15.31:0/2008078295 wait complete. 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 -- start start 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.397+0000 7f70c9301740 1 Processor -- start 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c9301740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912d4a0 0x561739138020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c9301740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912ddc0 0x5617391352d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c9301740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739135870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c9301740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56173913b250 con 0x561739138680 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c9301740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56173904e050 con 0x56173912ddc0 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c9301740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56173913b3d0 con 0x56173912d4a0 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c9301740 10 client.0 _ll_register_callbacks cb 0x56173908e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c54c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c54c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739135870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36564/0 (socket says 172.21.15.31:36564) 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c54c5640 1 -- 172.21.15.31:0/3762924248 learned_addr learned my addr 172.21.15.31:0/3762924248 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70bffff640 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912ddc0 0x5617391352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912d4a0 0x561739138020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70bffff640 1 -- 172.21.15.31:0/3762924248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912d4a0 msgr2=0x561739138020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70bffff640 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912d4a0 0x561739138020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70bffff640 1 -- 172.21.15.31:0/3762924248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 msgr2=0x561739135870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70bffff640 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739135870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70bffff640 1 -- 172.21.15.31:0/3762924248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561738e043c0 con 0x56173912ddc0 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c54c5640 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70bffff640 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912ddc0 0x5617391352d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56173912adb0 tx=0x7f70ac098400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=273e2bc2285d97a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f709ffff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f709ffff640 1 -- 172.21.15.31:0/3762924248 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac09bac0 con 0x56173912ddc0 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f709ffff640 1 -- 172.21.15.31:0/3762924248 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f70ac09b5b0 con 0x56173912ddc0 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.398+0000 7f70c9301740 1 -- 172.21.15.31:0/3762924248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b8036070 con 0x56173912ddc0 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f709ffff640 1 -- 172.21.15.31:0/3762924248 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f70ac09c860 con 0x56173912ddc0 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f70c9301740 10 client.25184 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f70c9301740 1 -- 172.21.15.31:0/3762924248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561739146bd0 con 0x56173912ddc0 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f709ffff640 1 -- 172.21.15.31:0/3762924248 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b8036070 con 0x56173912ddc0 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f709ffff640 1 -- 172.21.15.31:0/3762924248 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f70ac0a0170 con 0x56173912ddc0 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f70c9301740 10 client.25184 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f70c9301740 10 client.25184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f70c9301740 20 client.25184 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f70c9301740 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561739148090 0x56173914a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f70c9301740 4 client.25184 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f70c9301740 1 -- 172.21.15.31:0/3762924248 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617390a0850 con 0x561739148090 2024-09-16T00:42:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.399+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561739148090 0x56173914a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.400+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561739148090 0x56173914a480 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b275c8fddc0cbe8 server_cookie=60a33a9cf62eb21b in_seq=0 out_seq=0 2024-09-16T00:42:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.400+0000 7f709ffff640 10 client.25184 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.406+0000 7f709ffff640 1 -- 172.21.15.31:0/3762924248 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617390a0850 con 0x561739148090 2024-09-16T00:42:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.406+0000 7f709ffff640 10 client.25184 handle_command_reply: tid=0 2024-09-16T00:42:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.406+0000 7f70c9301740 1 client.25184 shutdown 2024-09-16T00:42:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 -- 172.21.15.31:0/3762924248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912ddc0 msgr2=0x5617391352d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912ddc0 0x5617391352d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56173912adb0 tx=0x7f70ac098400 comp rx=0 tx=0).stop 2024-09-16T00:42:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 -- 172.21.15.31:0/3762924248 shutdown_connections 2024-09-16T00:42:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912ddc0 0x5617391352d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561739148090 0x56173914a480 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 --2- 172.21.15.31:0/3762924248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912d4a0 0x561739138020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f709ffff640 0 client.25184 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 -- 172.21.15.31:0/3762924248 >> 172.21.15.31:0/3762924248 conn(0x561739129a30 msgr2=0x561739128b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 -- 172.21.15.31:0/3762924248 shutdown_connections 2024-09-16T00:42:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 1 -- 172.21.15.31:0/3762924248 wait complete. 2024-09-16T00:42:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.407+0000 7f70c9301740 20 client.25184 trim_cache size 0 max 16384 2024-09-16T00:42:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c9301740 1 Processor -- start 2024-09-16T00:42:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c9301740 1 -- start start 2024-09-16T00:42:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c9301740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173912d4a0 0x5617391444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c9301740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x5617391307f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c9301740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561739138680 0x56173914fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c9301740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01acc40 con 0x56173912d4a0 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c9301740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01af4c0 con 0x561739138680 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c9301740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01ad2b0 con 0x56173912ddc0 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c4cc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173912d4a0 0x5617391444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c4cc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173912d4a0 0x5617391444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36574/0 (socket says 172.21.15.31:36574) 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.424+0000 7f70c4cc4640 1 -- 172.21.15.31:0/2731408477 learned_addr learned my addr 172.21.15.31:0/2731408477 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70bffff640 1 --2- 172.21.15.31:0/2731408477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x5617391307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70c54c5640 1 --2- 172.21.15.31:0/2731408477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561739138680 0x56173914fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70c4cc4640 1 -- 172.21.15.31:0/2731408477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 msgr2=0x5617391307f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2731408477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x5617391307f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70c4cc4640 1 -- 172.21.15.31:0/2731408477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561739138680 msgr2=0x56173914fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2731408477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561739138680 0x56173914fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70c4cc4640 1 -- 172.21.15.31:0/2731408477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561739137c40 con 0x56173912d4a0 2024-09-16T00:42:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2731408477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173912d4a0 0x5617391444b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f70b4010c20 tx=0x7f70b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb7fb5426cb5580 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70aa7fc640 1 -- 172.21.15.31:0/2731408477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b401c040 con 0x56173912d4a0 2024-09-16T00:42:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70aa7fc640 1 -- 172.21.15.31:0/2731408477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401d040 con 0x56173912d4a0 2024-09-16T00:42:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.425+0000 7f70aa7fc640 1 -- 172.21.15.31:0/2731408477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b400f040 con 0x56173912d4a0 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 -- 172.21.15.31:0/2731408477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173912d4a0 msgr2=0x5617391444b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 --2- 172.21.15.31:0/2731408477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173912d4a0 0x5617391444b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f70b4010c20 tx=0x7f70b4013000 comp rx=0 tx=0).stop 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 -- 172.21.15.31:0/2731408477 shutdown_connections 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 --2- 172.21.15.31:0/2731408477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561739138680 0x56173914fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 --2- 172.21.15.31:0/2731408477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x5617391307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 --2- 172.21.15.31:0/2731408477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173912d4a0 0x5617391444b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 -- 172.21.15.31:0/2731408477 >> 172.21.15.31:0/2731408477 conn(0x561739129a30 msgr2=0x5617391300e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 -- 172.21.15.31:0/2731408477 shutdown_connections 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 -- 172.21.15.31:0/2731408477 wait complete. 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 -- start start 2024-09-16T00:42:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.426+0000 7f70c9301740 1 Processor -- start 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c9301740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x561739132380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c9301740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x5617391328c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c9301740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739132e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c9301740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01ad980 con 0x561739138680 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c9301740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01aedf0 con 0x56173912d4a0 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c9301740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01ae050 con 0x56173912ddc0 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c54c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739132e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c9301740 10 client.0 _ll_register_callbacks cb 0x7f70ac009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c4cc4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x561739132380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c54c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739132e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36586/0 (socket says 172.21.15.31:36586) 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c54c5640 1 -- 172.21.15.31:0/2668390464 learned_addr learned my addr 172.21.15.31:0/2668390464 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70bffff640 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x5617391328c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c4cc4640 1 -- 172.21.15.31:0/2668390464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 msgr2=0x5617391328c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x5617391328c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c4cc4640 1 -- 172.21.15.31:0/2668390464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 msgr2=0x561739132e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739132e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c4cc4640 1 -- 172.21.15.31:0/2668390464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617391b1d50 con 0x56173912d4a0 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.427+0000 7f70c54c5640 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739132e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c4cc4640 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x561739132380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004810 tx=0x7f70b4010f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25fb7bdbab13ca35 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f709ffff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f709ffff640 1 -- 172.21.15.31:0/2668390464 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4036070 con 0x56173912d4a0 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f709ffff640 1 -- 172.21.15.31:0/2668390464 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f70b4023840 con 0x56173912d4a0 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c9301740 1 -- 172.21.15.31:0/2668390464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b401d040 con 0x56173912d4a0 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f709ffff640 1 -- 172.21.15.31:0/2668390464 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f70b401c050 con 0x56173912d4a0 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c9301740 10 client.25190 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c9301740 1 -- 172.21.15.31:0/2668390464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5617391b3020 con 0x56173912d4a0 2024-09-16T00:42:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f709ffff640 1 -- 172.21.15.31:0/2668390464 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401d040 con 0x56173912d4a0 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f709ffff640 1 -- 172.21.15.31:0/2668390464 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f70b40323f0 con 0x56173912d4a0 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c9301740 10 client.25190 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c9301740 10 client.25190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c9301740 20 client.25190 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c9301740 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5617391b3710 0x5617391b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c9301740 4 client.25190 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.428+0000 7f70c9301740 1 -- 172.21.15.31:0/2668390464 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56173909ad60 con 0x5617391b3710 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.429+0000 7f70bffff640 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5617391b3710 0x5617391b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.429+0000 7f70bffff640 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5617391b3710 0x5617391b5b30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcfff1e21ac5a208 server_cookie=e66d9c94b2d2feec in_seq=0 out_seq=0 2024-09-16T00:42:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.429+0000 7f709ffff640 10 client.25190 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f709ffff640 1 -- 172.21.15.31:0/2668390464 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56173909ad60 con 0x5617391b3710 2024-09-16T00:42:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f709ffff640 10 client.25190 handle_command_reply: tid=0 2024-09-16T00:42:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 client.25190 shutdown 2024-09-16T00:42:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 -- 172.21.15.31:0/2668390464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 msgr2=0x561739132380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x561739132380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004810 tx=0x7f70b4010f80 comp rx=0 tx=0).stop 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 -- 172.21.15.31:0/2668390464 shutdown_connections 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561739138680 0x561739132e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5617391b3710 0x5617391b5b30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56173912ddc0 0x5617391328c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 --2- 172.21.15.31:0/2668390464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56173912d4a0 0x561739132380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f709ffff640 0 client.25190 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 -- 172.21.15.31:0/2668390464 >> 172.21.15.31:0/2668390464 conn(0x561739129a30 msgr2=0x5617391308e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 -- 172.21.15.31:0/2668390464 shutdown_connections 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 1 -- 172.21.15.31:0/2668390464 wait complete. 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.434+0000 7f70c9301740 20 client.25190 trim_cache size 0 max 16384 2024-09-16T00:42:38.435 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:42:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:42:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:42:38.436 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:42:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 -- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f70b00a3240 msgr2=0x7f70b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f70b00a3240 0x7f70b00a5700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x56173912a890 tx=0x7f70b4008040 comp rx=0 tx=0).stop 2024-09-16T00:42:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 -- 172.21.15.31:0/2113552228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70c0103740 msgr2=0x7f70c0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70c0103740 0x7f70c0198af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e9c0 tx=0x7f70ac00edc0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 -- 172.21.15.31:0/2113552228 shutdown_connections 2024-09-16T00:42:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70c0108290 0x7f70c019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70c0103740 0x7f70c0198af0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f70b00a3240 0x7f70b00a5700 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 --2- 172.21.15.31:0/2113552228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70c0102d90 0x7f70c01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 -- 172.21.15.31:0/2113552228 >> 172.21.15.31:0/2113552228 conn(0x7f70c00fc7d0 msgr2=0x7f70c0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 -- 172.21.15.31:0/2113552228 shutdown_connections 2024-09-16T00:42:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:38.435+0000 7f70c6f28640 1 -- 172.21.15.31:0/2113552228 wait complete. 2024-09-16T00:42:39.096 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:42:39.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:42:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:39.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:39 smithi031 ceph-mon[30956]: pgmap v401: 129 pgs: 129 active+clean; 5.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 497 KiB/s wr, 45 op/s 2024-09-16T00:42:39.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:39 smithi081 ceph-mon[35539]: pgmap v401: 129 pgs: 129 active+clean; 5.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 497 KiB/s wr, 45 op/s 2024-09-16T00:42:39.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:39 smithi102 ceph-mon[35364]: pgmap v401: 129 pgs: 129 active+clean; 5.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 497 KiB/s wr, 45 op/s 2024-09-16T00:42:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 -- 172.21.15.31:0/3792503116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c102d90 msgr2=0x7f8a6c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 --2- 172.21.15.31:0/3792503116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c102d90 0x7f8a6c103170 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009930 tx=0x7f8a5c02a420 comp rx=0 tx=0).stop 2024-09-16T00:42:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 -- 172.21.15.31:0/3792503116 shutdown_connections 2024-09-16T00:42:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 --2- 172.21.15.31:0/3792503116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a6c108290 0x7f8a6c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 --2- 172.21.15.31:0/3792503116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a6c103740 0x7f8a6c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 --2- 172.21.15.31:0/3792503116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c102d90 0x7f8a6c103170 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 -- 172.21.15.31:0/3792503116 >> 172.21.15.31:0/3792503116 conn(0x7f8a6c0fc7d0 msgr2=0x7f8a6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 -- 172.21.15.31:0/3792503116 shutdown_connections 2024-09-16T00:42:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 -- 172.21.15.31:0/3792503116 wait complete. 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.938+0000 7f8a7107e640 1 Processor -- start 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a7107e640 1 -- start start 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a7107e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a6c102d90 0x7f8a6c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a7107e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c103740 0x7f8a6c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a7107e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a6c108290 0x7f8a6c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a7107e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c112090 con 0x7f8a6c103740 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a7107e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c111f10 con 0x7f8a6c102d90 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a7107e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c112210 con 0x7f8a6c108290 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c103740 0x7f8a6c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a6c102d90 0x7f8a6c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a6c102d90 0x7f8a6c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:35304/0 (socket says 172.21.15.31:35304) 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6ad9d640 1 -- 172.21.15.31:0/2914598190 learned_addr learned my addr 172.21.15.31:0/2914598190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6b59e640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a6c108290 0x7f8a6c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6a59c640 1 -- 172.21.15.31:0/2914598190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a6c108290 msgr2=0x7f8a6c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6a59c640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a6c108290 0x7f8a6c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6a59c640 1 -- 172.21.15.31:0/2914598190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a6c102d90 msgr2=0x7f8a6c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6a59c640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a6c102d90 0x7f8a6c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6a59c640 1 -- 172.21.15.31:0/2914598190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c19d550 con 0x7f8a6c103740 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6ad9d640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a6c102d90 0x7f8a6c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.939+0000 7f8a6b59e640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a6c108290 0x7f8a6c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.940+0000 7f8a6a59c640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c103740 0x7f8a6c198ae0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400d6a0 tx=0x7f8a5400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.940+0000 7f8a63fff640 1 -- 172.21.15.31:0/2914598190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54017070 con 0x7f8a6c103740 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.940+0000 7f8a63fff640 1 -- 172.21.15.31:0/2914598190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a54009820 con 0x7f8a6c103740 2024-09-16T00:42:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.940+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c19d840 con 0x7f8a6c103740 2024-09-16T00:42:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.940+0000 7f8a63fff640 1 -- 172.21.15.31:0/2914598190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54004d00 con 0x7f8a6c103740 2024-09-16T00:42:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.940+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6c1a5120 con 0x7f8a6c103740 2024-09-16T00:42:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.941+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6c06a7f0 con 0x7f8a6c103740 2024-09-16T00:42:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.942+0000 7f8a63fff640 1 -- 172.21.15.31:0/2914598190 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8a540099d0 con 0x7f8a6c103740 2024-09-16T00:42:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.943+0000 7f8a63fff640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8a580a32c0 0x7f8a580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.943+0000 7f8a6ad9d640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8a580a32c0 0x7f8a580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.943+0000 7f8a63fff640 1 -- 172.21.15.31:0/2914598190 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f8a540cf5c0 con 0x7f8a6c103740 2024-09-16T00:42:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.944+0000 7f8a6ad9d640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8a580a32c0 0x7f8a580a5780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:39.947+0000 7f8a63fff640 1 -- 172.21.15.31:0/2914598190 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a5409bf30 con 0x7f8a6c103740 2024-09-16T00:42:40.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.168+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a6c10a2a0 con 0x7f8a6c103740 2024-09-16T00:42:40.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.168+0000 7f8a63fff640 1 -- 172.21.15.31:0/2914598190 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f8a5409b830 con 0x7f8a6c103740 2024-09-16T00:42:40.169 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:42:40.169 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:42:40.169 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:42:40.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8a580a32c0 msgr2=0x7f8a580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:40.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8a580a32c0 0x7f8a580a5780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c009560 comp rx=0 tx=0).stop 2024-09-16T00:42:40.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c103740 msgr2=0x7f8a6c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:40.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c103740 0x7f8a6c198ae0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400d6a0 tx=0x7f8a5400db60 comp rx=0 tx=0).stop 2024-09-16T00:42:40.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 shutdown_connections 2024-09-16T00:42:40.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8a580a32c0 0x7f8a580a5780 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:40.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a6c108290 0x7f8a6c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:40.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6c103740 0x7f8a6c198ae0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:40.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 --2- 172.21.15.31:0/2914598190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a6c102d90 0x7f8a6c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:40.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.170+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 >> 172.21.15.31:0/2914598190 conn(0x7f8a6c0fc7d0 msgr2=0x7f8a6c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:40.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.171+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 shutdown_connections 2024-09-16T00:42:40.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:40.171+0000 7f8a7107e640 1 -- 172.21.15.31:0/2914598190 wait complete. 2024-09-16T00:42:40.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:40 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2914598190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:40.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:40 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2914598190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:40.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:40 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2914598190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:42:40.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:42:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:41.547 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:41 smithi031 ceph-mon[30956]: pgmap v402: 129 pgs: 129 active+clean; 6.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 551 KiB/s wr, 51 op/s 2024-09-16T00:42:41.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:41 smithi102 ceph-mon[35364]: pgmap v402: 129 pgs: 129 active+clean; 6.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 551 KiB/s wr, 51 op/s 2024-09-16T00:42:41.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:41 smithi081 ceph-mon[35539]: pgmap v402: 129 pgs: 129 active+clean; 6.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 551 KiB/s wr, 51 op/s 2024-09-16T00:42:41.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.750+0000 7fb669407640 1 -- 172.21.15.31:0/2467753201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664102570 msgr2=0x7fb6641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:41.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.750+0000 7fb669407640 1 --2- 172.21.15.31:0/2467753201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664102570 0x7fb6641029b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb654009860 tx=0x7fb65402a410 comp rx=0 tx=0).stop 2024-09-16T00:42:41.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.756+0000 7fb669407640 1 -- 172.21.15.31:0/2467753201 shutdown_connections 2024-09-16T00:42:41.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.756+0000 7fb669407640 1 --2- 172.21.15.31:0/2467753201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb664102ef0 0x7fb664069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.756+0000 7fb669407640 1 --2- 172.21.15.31:0/2467753201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664102570 0x7fb6641029b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.756+0000 7fb669407640 1 --2- 172.21.15.31:0/2467753201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb664100e20 0x7fb664101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.756+0000 7fb669407640 1 -- 172.21.15.31:0/2467753201 >> 172.21.15.31:0/2467753201 conn(0x7fb6640fc7d0 msgr2=0x7fb6640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:41.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.760+0000 7fb669407640 1 -- 172.21.15.31:0/2467753201 shutdown_connections 2024-09-16T00:42:41.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.761+0000 7fb669407640 1 -- 172.21.15.31:0/2467753201 wait complete. 2024-09-16T00:42:41.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.762+0000 7fb669407640 1 Processor -- start 2024-09-16T00:42:41.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.765+0000 7fb669407640 1 -- start start 2024-09-16T00:42:41.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.766+0000 7fb669407640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664100e20 0x7fb664113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.766+0000 7fb669407640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb664102570 0x7fb664110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.766+0000 7fb669407640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb664102ef0 0x7fb664110f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.766+0000 7fb669407640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb664114390 con 0x7fb664102570 2024-09-16T00:42:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.766+0000 7fb669407640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb664114210 con 0x7fb664102ef0 2024-09-16T00:42:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.766+0000 7fb669407640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb664114510 con 0x7fb664100e20 2024-09-16T00:42:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.768+0000 7fb662ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664100e20 0x7fb664113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.768+0000 7fb662ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664100e20 0x7fb664113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:47924/0 (socket says 172.21.15.31:47924) 2024-09-16T00:42:41.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.768+0000 7fb662ffd640 1 -- 172.21.15.31:0/3727223868 learned_addr learned my addr 172.21.15.31:0/3727223868 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:41.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.769+0000 7fb6627fc640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb664102570 0x7fb664110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.770+0000 7fb662ffd640 1 -- 172.21.15.31:0/3727223868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb664102ef0 msgr2=0x7fb664110f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:42:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.770+0000 7fb662ffd640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb664102ef0 0x7fb664110f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.770+0000 7fb662ffd640 1 -- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb664102570 msgr2=0x7fb664110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.770+0000 7fb662ffd640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb664102570 0x7fb664110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.770+0000 7fb662ffd640 1 -- 172.21.15.31:0/3727223868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb664111800 con 0x7fb664100e20 2024-09-16T00:42:41.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.770+0000 7fb662ffd640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664100e20 0x7fb664113980 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb65800e9c0 tx=0x7fb65800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:41.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.771+0000 7fb64bfff640 1 -- 172.21.15.31:0/3727223868 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65800ccc0 con 0x7fb664100e20 2024-09-16T00:42:41.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.771+0000 7fb64bfff640 1 -- 172.21.15.31:0/3727223868 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb658005500 con 0x7fb664100e20 2024-09-16T00:42:41.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.771+0000 7fb64bfff640 1 -- 172.21.15.31:0/3727223868 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658010430 con 0x7fb664100e20 2024-09-16T00:42:41.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.771+0000 7fb669407640 1 -- 172.21.15.31:0/3727223868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66406bcd0 con 0x7fb664100e20 2024-09-16T00:42:41.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.771+0000 7fb669407640 1 -- 172.21.15.31:0/3727223868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66406c190 con 0x7fb664100e20 2024-09-16T00:42:41.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.776+0000 7fb66b7e0740 1 Processor -- start 2024-09-16T00:42:41.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.776+0000 7fb66b7e0740 1 -- start start 2024-09-16T00:42:41.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.777+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.777+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.778+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 0x55d8ab3b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.778+0000 7fb6637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.778+0000 7fb6637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36640/0 (socket says 172.21.15.31:36640) 2024-09-16T00:42:41.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.778+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d8ab30b2f0 con 0x55d8ab3a6b40 2024-09-16T00:42:41.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.778+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d8ab2b2cf0 con 0x55d8ab3a55d0 2024-09-16T00:42:41.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.778+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d8ab3072b0 con 0x55d8ab3b0370 2024-09-16T00:42:41.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.779+0000 7fb6637fe640 1 -- 172.21.15.31:0/1119011921 learned_addr learned my addr 172.21.15.31:0/1119011921 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:41.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.779+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1119011921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 0x55d8ab3b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.779+0000 7fb6627fc640 1 -- 172.21.15.31:0/1119011921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 msgr2=0x55d8ab3a5a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:42:41.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.779+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1119011921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3a5a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.779+0000 7fb6627fc640 1 -- 172.21.15.31:0/1119011921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 msgr2=0x55d8ab3a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:41.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.779+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1119011921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.779+0000 7fb6627fc640 1 -- 172.21.15.31:0/1119011921 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ab2973a0 con 0x55d8ab3b0370 2024-09-16T00:42:41.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.780+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1119011921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 0x55d8ab3b2760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb654009830 tx=0x7fb654030a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceb5e914d6de8398 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:41.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.782+0000 7fb6497fa640 1 -- 172.21.15.31:0/1119011921 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654030df0 con 0x55d8ab3b0370 2024-09-16T00:42:41.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.782+0000 7fb6497fa640 1 -- 172.21.15.31:0/1119011921 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb654040070 con 0x55d8ab3b0370 2024-09-16T00:42:41.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.783+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1119011921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 msgr2=0x55d8ab3b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:41.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.783+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1119011921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 0x55d8ab3b2760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb654009830 tx=0x7fb654030a80 comp rx=0 tx=0).stop 2024-09-16T00:42:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.783+0000 7fb64bfff640 1 -- 172.21.15.31:0/3727223868 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb6580046c0 con 0x7fb664100e20 2024-09-16T00:42:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.784+0000 7fb64bfff640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb64c0a3450 0x7fb64c0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.785+0000 7fb6637fe640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb64c0a3450 0x7fb64c0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.785+0000 7fb6637fe640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb64c0a3450 0x7fb64c0a5910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009e40 tx=0x7fb65c0094c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.785+0000 7fb64bfff640 1 -- 172.21.15.31:0/3727223868 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb6580d00e0 con 0x7fb664100e20 2024-09-16T00:42:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.786+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1119011921 shutdown_connections 2024-09-16T00:42:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.786+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1119011921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 0x55d8ab3b2760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.786+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1119011921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.786+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1119011921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.786+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1119011921 >> 172.21.15.31:0/1119011921 conn(0x55d8ab36e310 msgr2=0x55d8ab3a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.791+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1119011921 shutdown_connections 2024-09-16T00:42:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.792+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1119011921 wait complete. 2024-09-16T00:42:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.792+0000 7fb66b7e0740 1 -- start start 2024-09-16T00:42:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.793+0000 7fb66b7e0740 1 Processor -- start 2024-09-16T00:42:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.793+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.793+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.793+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3b0370 0x55d8ab3ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.793+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d8ab3b3060 con 0x55d8ab3b0370 2024-09-16T00:42:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.793+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d8ab2c6060 con 0x55d8ab3a6b40 2024-09-16T00:42:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.793+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d8ab3b31e0 con 0x55d8ab3a55d0 2024-09-16T00:42:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.793+0000 7fb66b7e0740 10 client.0 _ll_register_callbacks cb 0x55d8ab0198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:41.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.794+0000 7fb6637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3b0370 0x55d8ab3ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.796+0000 7fb6637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3b0370 0x55d8ab3ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36652/0 (socket says 172.21.15.31:36652) 2024-09-16T00:42:41.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.796+0000 7fb6637fe640 1 -- 172.21.15.31:0/2854567250 learned_addr learned my addr 172.21.15.31:0/2854567250 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:41.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.796+0000 7fb6627fc640 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.797+0000 7fb662ffd640 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.798+0000 7fb6627fc640 1 -- 172.21.15.31:0/2854567250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a55d0 msgr2=0x55d8ab3afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.798+0000 7fb6627fc640 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.798+0000 7fb6627fc640 1 -- 172.21.15.31:0/2854567250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3b0370 msgr2=0x55d8ab3ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.798+0000 7fb6627fc640 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3b0370 0x55d8ab3ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.798+0000 7fb6627fc640 1 -- 172.21.15.31:0/2854567250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ab07c3c0 con 0x55d8ab3a6b40 2024-09-16T00:42:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.798+0000 7fb6627fc640 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3ad000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb654009780 tx=0x7fb6540305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a454bffedec4430 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.798+0000 7fb646ff5640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:42:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.799+0000 7fb646ff5640 1 -- 172.21.15.31:0/2854567250 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654004b00 con 0x55d8ab3a6b40 2024-09-16T00:42:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.799+0000 7fb66b7e0740 1 -- 172.21.15.31:0/2854567250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb654040070 con 0x55d8ab3a6b40 2024-09-16T00:42:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.800+0000 7fb646ff5640 1 -- 172.21.15.31:0/2854567250 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb654004ca0 con 0x55d8ab3a6b40 2024-09-16T00:42:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.801+0000 7fb646ff5640 1 -- 172.21.15.31:0/2854567250 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb65405a050 con 0x55d8ab3a6b40 2024-09-16T00:42:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb66b7e0740 10 client.25199 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb66b7e0740 1 -- 172.21.15.31:0/2854567250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d8ab3beb30 con 0x55d8ab3a6b40 2024-09-16T00:42:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb646ff5640 1 -- 172.21.15.31:0/2854567250 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb654040070 con 0x55d8ab3a6b40 2024-09-16T00:42:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb646ff5640 1 -- 172.21.15.31:0/2854567250 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb65402fbd0 con 0x55d8ab3a6b40 2024-09-16T00:42:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb66b7e0740 10 client.25199 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb66b7e0740 10 client.25199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb66b7e0740 20 client.25199 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8ab3c00d0 0x55d8ab3c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb66b7e0740 4 client.25199 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.802+0000 7fb66b7e0740 1 -- 172.21.15.31:0/2854567250 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8ab318860 con 0x55d8ab3c00d0 2024-09-16T00:42:41.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.805+0000 7fb662ffd640 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8ab3c00d0 0x55d8ab3c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.808+0000 7fb662ffd640 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8ab3c00d0 0x55d8ab3c24c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f94c118bfea267 server_cookie=d85413d80e78b2d3 in_seq=0 out_seq=0 2024-09-16T00:42:41.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.808+0000 7fb646ff5640 10 client.25199 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:41.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.811+0000 7fb646ff5640 1 -- 172.21.15.31:0/2854567250 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8ab318860 con 0x55d8ab3c00d0 2024-09-16T00:42:41.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.811+0000 7fb646ff5640 10 client.25199 handle_command_reply: tid=0 2024-09-16T00:42:41.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.811+0000 7fb66b7e0740 1 client.25199 shutdown 2024-09-16T00:42:41.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.813+0000 7fb66b7e0740 1 -- 172.21.15.31:0/2854567250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a6b40 msgr2=0x55d8ab3ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:41.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.813+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3ad000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb654009780 tx=0x7fb6540305d0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.815+0000 7fb66b7e0740 1 -- 172.21.15.31:0/2854567250 shutdown_connections 2024-09-16T00:42:41.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.816+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3b0370 0x55d8ab3ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.816+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3ad000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.816+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8ab3c00d0 0x55d8ab3c24c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.816+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/2854567250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.816+0000 7fb66b7e0740 1 -- 172.21.15.31:0/2854567250 >> 172.21.15.31:0/2854567250 conn(0x55d8ab36e310 msgr2=0x55d8ab3a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:41.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.816+0000 7fb646ff5640 0 client.25199 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:41.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.817+0000 7fb66b7e0740 1 -- 172.21.15.31:0/2854567250 shutdown_connections 2024-09-16T00:42:41.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.820+0000 7fb66b7e0740 1 -- 172.21.15.31:0/2854567250 wait complete. 2024-09-16T00:42:41.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.821+0000 7fb66b7e0740 20 client.25199 trim_cache size 0 max 16384 2024-09-16T00:42:41.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.850+0000 7fb66b7e0740 1 Processor -- start 2024-09-16T00:42:41.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.851+0000 7fb66b7e0740 1 -- start start 2024-09-16T00:42:41.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.851+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.851+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.852+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 0x55d8ab3c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.852+0000 7fb662ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.852+0000 7fb662ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:35334/0 (socket says 172.21.15.31:35334) 2024-09-16T00:42:41.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.852+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6641029b0 con 0x55d8ab3a6b40 2024-09-16T00:42:41.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.852+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb6641be420 con 0x55d8ab3a55d0 2024-09-16T00:42:41.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.852+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb664073d20 con 0x55d8ab3b0370 2024-09-16T00:42:41.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.853+0000 7fb6627fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.853+0000 7fb6627fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3bc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36662/0 (socket says 172.21.15.31:36662) 2024-09-16T00:42:41.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.853+0000 7fb6627fc640 1 -- 172.21.15.31:0/1328857273 learned_addr learned my addr 172.21.15.31:0/1328857273 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:41.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.854+0000 7fb6637fe640 1 --2- 172.21.15.31:0/1328857273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 0x55d8ab3c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.854+0000 7fb6627fc640 1 -- 172.21.15.31:0/1328857273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 msgr2=0x55d8ab3c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:41.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.854+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1328857273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 0x55d8ab3c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.854+0000 7fb6627fc640 1 -- 172.21.15.31:0/1328857273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 msgr2=0x55d8ab3a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.854+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1328857273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.854+0000 7fb6627fc640 1 -- 172.21.15.31:0/1328857273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ab3a8120 con 0x55d8ab3a6b40 2024-09-16T00:42:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.855+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1328857273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3bc350 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb65402f470 tx=0x7fb654000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed8dcd243b1a8c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.861+0000 7fb648ff9640 1 -- 172.21.15.31:0/1328857273 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654049070 con 0x55d8ab3a6b40 2024-09-16T00:42:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.861+0000 7fb648ff9640 1 -- 172.21.15.31:0/1328857273 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb654040040 con 0x55d8ab3a6b40 2024-09-16T00:42:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.861+0000 7fb648ff9640 1 -- 172.21.15.31:0/1328857273 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65402fa40 con 0x55d8ab3a6b40 2024-09-16T00:42:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.863+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1328857273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 msgr2=0x55d8ab3bc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.863+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1328857273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3bc350 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb65402f470 tx=0x7fb654000bc0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.866+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1328857273 shutdown_connections 2024-09-16T00:42:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.866+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1328857273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3b0370 0x55d8ab3c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.866+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1328857273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3bc350 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.866+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1328857273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.866+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1328857273 >> 172.21.15.31:0/1328857273 conn(0x55d8ab36e310 msgr2=0x55d8ab3a1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:41.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.868+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1328857273 shutdown_connections 2024-09-16T00:42:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.869+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1328857273 wait complete. 2024-09-16T00:42:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.869+0000 7fb66b7e0740 1 -- start start 2024-09-16T00:42:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.870+0000 7fb66b7e0740 1 Processor -- start 2024-09-16T00:42:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.870+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.870+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.871+0000 7fb66b7e0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3b0370 0x55d8ab3aa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.871+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6641bc980 con 0x55d8ab3a55d0 2024-09-16T00:42:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.871+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb6641bdd50 con 0x55d8ab3b0370 2024-09-16T00:42:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.871+0000 7fb66b7e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb6641bcfb0 con 0x55d8ab3a6b40 2024-09-16T00:42:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.871+0000 7fb6637fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3b0370 0x55d8ab3aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.871+0000 7fb6627fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.871+0000 7fb6627fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:47948/0 (socket says 172.21.15.31:47948) 2024-09-16T00:42:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.871+0000 7fb6627fc640 1 -- 172.21.15.31:0/1346774994 learned_addr learned my addr 172.21.15.31:0/1346774994 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.871+0000 7fb66b7e0740 10 client.0 _ll_register_callbacks cb 0x7fb6540096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:41.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.872+0000 7fb662ffd640 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.873+0000 7fb6627fc640 1 -- 172.21.15.31:0/1346774994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3b0370 msgr2=0x55d8ab3aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.873+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3b0370 0x55d8ab3aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.873+0000 7fb6627fc640 1 -- 172.21.15.31:0/1346774994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a55d0 msgr2=0x55d8ab3b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.873+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.873+0000 7fb6627fc640 1 -- 172.21.15.31:0/1346774994 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ab3aab60 con 0x55d8ab3a6b40 2024-09-16T00:42:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.873+0000 7fb6627fc640 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a9dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb65402aa00 tx=0x7fb654009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bc26c6aa3af8d10 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.874+0000 7fb646ff5640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:42:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.874+0000 7fb646ff5640 1 -- 172.21.15.31:0/1346774994 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654049040 con 0x55d8ab3a6b40 2024-09-16T00:42:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.874+0000 7fb646ff5640 1 -- 172.21.15.31:0/1346774994 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb654045ca0 con 0x55d8ab3a6b40 2024-09-16T00:42:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.874+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1346774994 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb654040040 con 0x55d8ab3a6b40 2024-09-16T00:42:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.878+0000 7fb646ff5640 1 -- 172.21.15.31:0/1346774994 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb65402f560 con 0x55d8ab3a6b40 2024-09-16T00:42:41.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.878+0000 7fb66b7e0740 10 client.25159 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:41.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.878+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1346774994 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d8ab42ad60 con 0x55d8ab3a6b40 2024-09-16T00:42:41.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.879+0000 7fb646ff5640 1 -- 172.21.15.31:0/1346774994 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb65402fbe0 con 0x55d8ab3a6b40 2024-09-16T00:42:41.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.879+0000 7fb646ff5640 1 -- 172.21.15.31:0/1346774994 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb6540461d0 con 0x55d8ab3a6b40 2024-09-16T00:42:41.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.882+0000 7fb66b7e0740 10 client.25159 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:41.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.882+0000 7fb66b7e0740 10 client.25159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:41.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.882+0000 7fb66b7e0740 20 client.25159 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:41.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.882+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8ab42b2f0 0x55d8ab42d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:41.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.882+0000 7fb66b7e0740 4 client.25159 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:42:41.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.882+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1346774994 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb654040040 con 0x55d8ab42b2f0 2024-09-16T00:42:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.883+0000 7fb662ffd640 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8ab42b2f0 0x55d8ab42d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.886+0000 7fb662ffd640 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8ab42b2f0 0x55d8ab42d710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf9bd8bd8323b8 server_cookie=e972c214b4805905 in_seq=0 out_seq=0 2024-09-16T00:42:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.886+0000 7fb646ff5640 10 client.25159 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.888+0000 7fb646ff5640 1 -- 172.21.15.31:0/1346774994 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb654040040 con 0x55d8ab42b2f0 2024-09-16T00:42:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.888+0000 7fb646ff5640 10 client.25159 handle_command_reply: tid=0 2024-09-16T00:42:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.888+0000 7fb66b7e0740 1 client.25159 shutdown 2024-09-16T00:42:41.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.891+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1346774994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a6b40 msgr2=0x55d8ab3a9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:41.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.891+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a9dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb65402aa00 tx=0x7fb654009370 comp rx=0 tx=0).stop 2024-09-16T00:42:41.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.892+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1346774994 shutdown_connections 2024-09-16T00:42:41.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.892+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8ab3b0370 0x55d8ab3aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.892+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8ab42b2f0 0x55d8ab42d710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.892+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8ab3a6b40 0x55d8ab3a9dd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.892+0000 7fb66b7e0740 1 --2- 172.21.15.31:0/1346774994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8ab3a55d0 0x55d8ab3b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.892+0000 7fb646ff5640 0 client.25159 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:41.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.892+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1346774994 >> 172.21.15.31:0/1346774994 conn(0x55d8ab36e310 msgr2=0x55d8ab3a04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.893+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1346774994 shutdown_connections 2024-09-16T00:42:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.894+0000 7fb66b7e0740 1 -- 172.21.15.31:0/1346774994 wait complete. 2024-09-16T00:42:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.894+0000 7fb66b7e0740 20 client.25159 trim_cache size 0 max 16384 2024-09-16T00:42:41.895 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:42:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.895+0000 7fb669407640 1 -- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb64c0a3450 msgr2=0x7fb64c0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.895+0000 7fb669407640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb64c0a3450 0x7fb64c0a5910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009e40 tx=0x7fb65c0094c0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.895+0000 7fb669407640 1 -- 172.21.15.31:0/3727223868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664100e20 msgr2=0x7fb664113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.895+0000 7fb669407640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664100e20 0x7fb664113980 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb65800e9c0 tx=0x7fb65800ee80 comp rx=0 tx=0).stop 2024-09-16T00:42:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.895+0000 7fb669407640 1 -- 172.21.15.31:0/3727223868 shutdown_connections 2024-09-16T00:42:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.895+0000 7fb669407640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb664102ef0 0x7fb664110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.896+0000 7fb669407640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb664102570 0x7fb664110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.896+0000 7fb669407640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb64c0a3450 0x7fb64c0a5910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.896+0000 7fb669407640 1 --2- 172.21.15.31:0/3727223868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb664100e20 0x7fb664113980 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:41.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.896+0000 7fb669407640 1 -- 172.21.15.31:0/3727223868 >> 172.21.15.31:0/3727223868 conn(0x7fb6640fc7d0 msgr2=0x7fb6641067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:41.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.898+0000 7fb669407640 1 -- 172.21.15.31:0/3727223868 shutdown_connections 2024-09-16T00:42:41.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:41.898+0000 7fb669407640 1 -- 172.21.15.31:0/3727223868 wait complete. 2024-09-16T00:42:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:43 smithi102 ceph-mon[35364]: pgmap v403: 129 pgs: 129 active+clean; 7.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 734 KiB/s wr, 58 op/s 2024-09-16T00:42:43.610 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:42:43.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:42:43.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:43 smithi031 ceph-mon[30956]: pgmap v403: 129 pgs: 129 active+clean; 7.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 734 KiB/s wr, 58 op/s 2024-09-16T00:42:43.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:43 smithi081 ceph-mon[35539]: pgmap v403: 129 pgs: 129 active+clean; 7.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 734 KiB/s wr, 58 op/s 2024-09-16T00:42:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:44.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.434+0000 7f55d8416640 1 -- 172.21.15.31:0/4083025424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55d0102790 msgr2=0x7f55d01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:44.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.434+0000 7f55d8416640 1 --2- 172.21.15.31:0/4083025424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55d0102790 0x7f55d01067e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009900 tx=0x7f55c002a440 comp rx=0 tx=0).stop 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.436+0000 7f55d8416640 1 -- 172.21.15.31:0/4083025424 shutdown_connections 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.436+0000 7f55d8416640 1 --2- 172.21.15.31:0/4083025424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55d0106f10 0x7f55d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.436+0000 7f55d8416640 1 --2- 172.21.15.31:0/4083025424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55d0102790 0x7f55d01067e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.436+0000 7f55d8416640 1 --2- 172.21.15.31:0/4083025424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0101de0 0x7f55d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.436+0000 7f55d8416640 1 -- 172.21.15.31:0/4083025424 >> 172.21.15.31:0/4083025424 conn(0x7f55d00fd8c0 msgr2=0x7f55d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.438+0000 7f55d8416640 1 -- 172.21.15.31:0/4083025424 shutdown_connections 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.440+0000 7f55d8416640 1 -- 172.21.15.31:0/4083025424 wait complete. 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.441+0000 7f55d8416640 1 Processor -- start 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.441+0000 7f55d8416640 1 -- start start 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d8416640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0101de0 0x7f55d010cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d8416640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55d0102790 0x7f55d010d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d8416640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55d0106f10 0x7f55d010d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d8416640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55d010fd70 con 0x7f55d0102790 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d8416640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55d010fbf0 con 0x7f55d0106f10 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d8416640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55d010fef0 con 0x7f55d0101de0 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d59b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55d0102790 0x7f55d010d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d59b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55d0102790 0x7f55d010d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36700/0 (socket says 172.21.15.31:36700) 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d59b1640 1 -- 172.21.15.31:0/2168122832 learned_addr learned my addr 172.21.15.31:0/2168122832 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.442+0000 7f55d69b3640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55d0106f10 0x7f55d010d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.443+0000 7f55d61b2640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0101de0 0x7f55d010cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.443+0000 7f55d59b1640 1 -- 172.21.15.31:0/2168122832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0101de0 msgr2=0x7f55d010cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.443+0000 7f55d59b1640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0101de0 0x7f55d010cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.443+0000 7f55d59b1640 1 -- 172.21.15.31:0/2168122832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55d0106f10 msgr2=0x7f55d010d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:44.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.443+0000 7f55d59b1640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55d0106f10 0x7f55d010d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.443+0000 7f55d59b1640 1 -- 172.21.15.31:0/2168122832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d01aac80 con 0x7f55d0102790 2024-09-16T00:42:44.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.443+0000 7f55d61b2640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0101de0 0x7f55d010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:44.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.443+0000 7f55d59b1640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55d0102790 0x7f55d010d280 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009a20 tx=0x7f55c00052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:44.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.444+0000 7f55c77fe640 1 -- 172.21.15.31:0/2168122832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c003f070 con 0x7f55d0102790 2024-09-16T00:42:44.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.444+0000 7f55c77fe640 1 -- 172.21.15.31:0/2168122832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c0003cd0 con 0x7f55d0102790 2024-09-16T00:42:44.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.445+0000 7f55c77fe640 1 -- 172.21.15.31:0/2168122832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0035030 con 0x7f55d0102790 2024-09-16T00:42:44.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.446+0000 7f55d8416640 1 -- 172.21.15.31:0/2168122832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d01aaf10 con 0x7f55d0102790 2024-09-16T00:42:44.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.447+0000 7f55d8416640 1 -- 172.21.15.31:0/2168122832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d01ab2b0 con 0x7f55d0102790 2024-09-16T00:42:44.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.450+0000 7f55da7ef740 1 Processor -- start 2024-09-16T00:42:44.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.451+0000 7f55da7ef740 1 -- start start 2024-09-16T00:42:44.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.451+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 0x55f34da32f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.451+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da33480 0x55f34da3d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.451+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da3dd40 0x55f34da40130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.451+0000 7f55da7ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f34d998280 con 0x55f34da32b60 2024-09-16T00:42:44.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.451+0000 7f55da7ef740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f34d93fb10 con 0x55f34da3dd40 2024-09-16T00:42:44.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.451+0000 7f55da7ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f34d994240 con 0x55f34da33480 2024-09-16T00:42:44.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.452+0000 7f55d61b2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da33480 0x55f34da3d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.452+0000 7f55d61b2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da33480 0x55f34da3d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:47986/0 (socket says 172.21.15.31:47986) 2024-09-16T00:42:44.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.452+0000 7f55d61b2640 1 -- 172.21.15.31:0/842449674 learned_addr learned my addr 172.21.15.31:0/842449674 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:44.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.452+0000 7f55d69b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 0x55f34da32f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.453+0000 7f55d69b3640 1 -- 172.21.15.31:0/842449674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da33480 msgr2=0x55f34da3d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:44.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.453+0000 7f55d69b3640 1 --2- 172.21.15.31:0/842449674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da33480 0x55f34da3d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.453+0000 7f55d69b3640 1 -- 172.21.15.31:0/842449674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da3dd40 msgr2=0x55f34da40130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:42:44.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.453+0000 7f55d69b3640 1 --2- 172.21.15.31:0/842449674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da3dd40 0x55f34da40130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.453+0000 7f55d69b3640 1 -- 172.21.15.31:0/842449674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34d94c1f0 con 0x55f34da32b60 2024-09-16T00:42:44.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.453+0000 7f55d69b3640 1 --2- 172.21.15.31:0/842449674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 0x55f34da32f40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f55c8005fd0 tx=0x7f55c800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a565984cd039cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:44.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.454+0000 7f55c77fe640 1 -- 172.21.15.31:0/2168122832 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f55c0042070 con 0x7f55d0102790 2024-09-16T00:42:44.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.455+0000 7f55c77fe640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55bc0a31e0 0x7f55bc0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.456+0000 7f55c77fe640 1 -- 172.21.15.31:0/2168122832 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f55c00f08f0 con 0x7f55d0102790 2024-09-16T00:42:44.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.457+0000 7f55d61b2640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55bc0a31e0 0x7f55bc0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.457+0000 7f55d61b2640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55bc0a31e0 0x7f55bc0a56a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55f34da2ff50 tx=0x7f55cc0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:44.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.460+0000 7f55c4dea640 1 -- 172.21.15.31:0/842449674 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8017620 con 0x55f34da32b60 2024-09-16T00:42:44.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.460+0000 7f55c4dea640 1 -- 172.21.15.31:0/842449674 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c8017ca0 con 0x55f34da32b60 2024-09-16T00:42:44.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.461+0000 7f55da7ef740 1 -- 172.21.15.31:0/842449674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 msgr2=0x55f34da32f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:44.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.461+0000 7f55da7ef740 1 --2- 172.21.15.31:0/842449674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 0x55f34da32f40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f55c8005fd0 tx=0x7f55c800d710 comp rx=0 tx=0).stop 2024-09-16T00:42:44.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.462+0000 7f55da7ef740 1 -- 172.21.15.31:0/842449674 shutdown_connections 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.462+0000 7f55da7ef740 1 --2- 172.21.15.31:0/842449674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da3dd40 0x55f34da40130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.462+0000 7f55da7ef740 1 --2- 172.21.15.31:0/842449674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da33480 0x55f34da3d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.462+0000 7f55da7ef740 1 --2- 172.21.15.31:0/842449674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 0x55f34da32f40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.462+0000 7f55da7ef740 1 -- 172.21.15.31:0/842449674 >> 172.21.15.31:0/842449674 conn(0x55f34da2f0f0 msgr2=0x55f34da2f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.466+0000 7f55da7ef740 1 -- 172.21.15.31:0/842449674 shutdown_connections 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.469+0000 7f55da7ef740 1 -- 172.21.15.31:0/842449674 wait complete. 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.469+0000 7f55da7ef740 1 -- start start 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.469+0000 7f55da7ef740 1 Processor -- start 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 0x55f34da3abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da33480 0x55f34da3b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da3b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55da7ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f34da51970 con 0x55f34da32b60 2024-09-16T00:42:44.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55da7ef740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f34d952f10 con 0x55f34da33480 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55da7ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f34da51af0 con 0x55f34da3dd40 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55da7ef740 10 client.0 _ll_register_callbacks cb 0x55f34d91ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55d61b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 0x55f34da3abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55d69b3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da3b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55d69b3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da3b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:47990/0 (socket says 172.21.15.31:47990) 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55d69b3640 1 -- 172.21.15.31:0/1717594975 learned_addr learned my addr 172.21.15.31:0/1717594975 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.470+0000 7f55d59b1640 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da33480 0x55f34da3b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.471+0000 7f55d59b1640 1 -- 172.21.15.31:0/1717594975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 msgr2=0x55f34da3b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.471+0000 7f55d59b1640 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da3b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.471+0000 7f55d59b1640 1 -- 172.21.15.31:0/1717594975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 msgr2=0x55f34da3abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.471+0000 7f55d59b1640 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 0x55f34da3abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.471+0000 7f55d59b1640 1 -- 172.21.15.31:0/1717594975 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34d7096a0 con 0x55f34da33480 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.474+0000 7f55d59b1640 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da33480 0x55f34da3b130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55f34da30470 tx=0x7f55c00bc730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17cd42c2ea7f6f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.474+0000 7f55b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.478+0000 7f55b9ffb640 1 -- 172.21.15.31:0/1717594975 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c00bcb00 con 0x55f34da33480 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.478+0000 7f55da7ef740 1 -- 172.21.15.31:0/1717594975 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c8017ca0 con 0x55f34da33480 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.478+0000 7f55b9ffb640 1 -- 172.21.15.31:0/1717594975 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f55c00314d0 con 0x55f34da33480 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.478+0000 7f55b9ffb640 1 -- 172.21.15.31:0/1717594975 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f55c00c0eb0 con 0x55f34da33480 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.480+0000 7f55da7ef740 10 client.25211 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.480+0000 7f55da7ef740 1 -- 172.21.15.31:0/1717594975 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f34da51970 con 0x55f34da33480 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.486+0000 7f55b9ffb640 1 -- 172.21.15.31:0/1717594975 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c8017ca0 con 0x55f34da33480 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.487+0000 7f55b9ffb640 1 -- 172.21.15.31:0/1717594975 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f55c004c060 con 0x55f34da33480 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.488+0000 7f55da7ef740 10 client.25211 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.488+0000 7f55da7ef740 10 client.25211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.488+0000 7f55da7ef740 20 client.25211 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:44.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.488+0000 7f55da7ef740 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f34da43150 0x55f34da45570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.488+0000 7f55da7ef740 4 client.25211 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:44.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.488+0000 7f55da7ef740 1 -- 172.21.15.31:0/1717594975 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f34d9a57f0 con 0x55f34da43150 2024-09-16T00:42:44.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.491+0000 7f55d61b2640 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f34da43150 0x55f34da45570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.495+0000 7f55d61b2640 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f34da43150 0x55f34da45570 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40db1da15f813ca9 server_cookie=2dab0eed3c1649a6 in_seq=0 out_seq=0 2024-09-16T00:42:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.496+0000 7f55b9ffb640 10 client.25211 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:44.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.499+0000 7f55b9ffb640 1 -- 172.21.15.31:0/1717594975 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f34d9a57f0 con 0x55f34da43150 2024-09-16T00:42:44.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.499+0000 7f55b9ffb640 10 client.25211 handle_command_reply: tid=0 2024-09-16T00:42:44.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.499+0000 7f55da7ef740 1 client.25211 shutdown 2024-09-16T00:42:44.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.500+0000 7f55da7ef740 1 -- 172.21.15.31:0/1717594975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da33480 msgr2=0x55f34da3b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.500+0000 7f55da7ef740 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da33480 0x55f34da3b130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55f34da30470 tx=0x7f55c00bc730 comp rx=0 tx=0).stop 2024-09-16T00:42:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.500+0000 7f55da7ef740 1 -- 172.21.15.31:0/1717594975 shutdown_connections 2024-09-16T00:42:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.500+0000 7f55da7ef740 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f34da43150 0x55f34da45570 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.500+0000 7f55da7ef740 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da3b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.500+0000 7f55da7ef740 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da33480 0x55f34da3b130 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.500+0000 7f55da7ef740 1 --2- 172.21.15.31:0/1717594975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da32b60 0x55f34da3abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.500+0000 7f55da7ef740 1 -- 172.21.15.31:0/1717594975 >> 172.21.15.31:0/1717594975 conn(0x55f34da2f0f0 msgr2=0x55f34da34e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.500+0000 7f55b9ffb640 0 client.25211 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.501+0000 7f55da7ef740 1 -- 172.21.15.31:0/1717594975 shutdown_connections 2024-09-16T00:42:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.501+0000 7f55da7ef740 1 -- 172.21.15.31:0/1717594975 wait complete. 2024-09-16T00:42:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.501+0000 7f55da7ef740 20 client.25211 trim_cache size 0 max 16384 2024-09-16T00:42:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.519+0000 7f55da7ef740 1 Processor -- start 2024-09-16T00:42:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.519+0000 7f55da7ef740 1 -- start start 2024-09-16T00:42:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da50e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da3b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da3ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55da7ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55d011afe0 con 0x55f34da33480 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55da7ef740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55d01aced0 con 0x55f34da32b60 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55da7ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55d011b5a0 con 0x55f34da3dd40 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d59b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da3b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d61b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da50e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d61b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da50e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:35380/0 (socket says 172.21.15.31:35380) 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d59b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da3b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36742/0 (socket says 172.21.15.31:36742) 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d61b2640 1 -- 172.21.15.31:0/2044003099 learned_addr learned my addr 172.21.15.31:0/2044003099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d69b3640 1 --2- 172.21.15.31:0/2044003099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da3ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d61b2640 1 -- 172.21.15.31:0/2044003099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 msgr2=0x55f34da3ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d61b2640 1 --2- 172.21.15.31:0/2044003099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da3ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d61b2640 1 -- 172.21.15.31:0/2044003099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 msgr2=0x55f34da3b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d61b2640 1 --2- 172.21.15.31:0/2044003099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da3b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d61b2640 1 -- 172.21.15.31:0/2044003099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34da2ef20 con 0x55f34da32b60 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d59b1640 1 --2- 172.21.15.31:0/2044003099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da3b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.520+0000 7f55d61b2640 1 --2- 172.21.15.31:0/2044003099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da50e70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f55cc010c90 tx=0x7f55cc012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6715cecff558ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.521+0000 7f55bbfff640 1 -- 172.21.15.31:0/2044003099 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc01f110 con 0x55f34da32b60 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.521+0000 7f55bbfff640 1 -- 172.21.15.31:0/2044003099 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55cc01fe00 con 0x55f34da32b60 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.521+0000 7f55bbfff640 1 -- 172.21.15.31:0/2044003099 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc005440 con 0x55f34da32b60 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.521+0000 7f55da7ef740 1 -- 172.21.15.31:0/2044003099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 msgr2=0x55f34da50e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.521+0000 7f55da7ef740 1 --2- 172.21.15.31:0/2044003099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da50e70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f55cc010c90 tx=0x7f55cc012040 comp rx=0 tx=0).stop 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.521+0000 7f55da7ef740 1 -- 172.21.15.31:0/2044003099 shutdown_connections 2024-09-16T00:42:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.522+0000 7f55da7ef740 1 --2- 172.21.15.31:0/2044003099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da3ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.522+0000 7f55da7ef740 1 --2- 172.21.15.31:0/2044003099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da3b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.522+0000 7f55da7ef740 1 --2- 172.21.15.31:0/2044003099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da50e70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.522+0000 7f55da7ef740 1 -- 172.21.15.31:0/2044003099 >> 172.21.15.31:0/2044003099 conn(0x55f34da2f0f0 msgr2=0x55f34da3c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.522+0000 7f55da7ef740 1 -- 172.21.15.31:0/2044003099 shutdown_connections 2024-09-16T00:42:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.522+0000 7f55da7ef740 1 -- 172.21.15.31:0/2044003099 wait complete. 2024-09-16T00:42:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.522+0000 7f55da7ef740 1 -- start start 2024-09-16T00:42:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.522+0000 7f55da7ef740 1 Processor -- start 2024-09-16T00:42:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da52a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da52fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55da7ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55da7ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55d01ab480 con 0x55f34da33480 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55da7ef740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55d01ac800 con 0x55f34da32b60 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55da7ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55d01aba60 con 0x55f34da3dd40 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d61b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da52a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d61b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da52a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:35386/0 (socket says 172.21.15.31:35386) 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55da7ef740 10 client.0 _ll_register_callbacks cb 0x7f55c0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d61b2640 1 -- 172.21.15.31:0/3980278410 learned_addr learned my addr 172.21.15.31:0/3980278410 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d59b1640 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da52fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d69b3640 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d61b2640 1 -- 172.21.15.31:0/3980278410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 msgr2=0x55f34da534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d61b2640 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d61b2640 1 -- 172.21.15.31:0/3980278410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 msgr2=0x55f34da52fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d61b2640 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da52fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d61b2640 1 -- 172.21.15.31:0/3980278410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34dad8e80 con 0x55f34da32b60 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55d61b2640 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da52a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55cc000c00 tx=0x7f55cc00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42cc9c1b03083bec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.523+0000 7f55b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55b9ffb640 1 -- 172.21.15.31:0/3980278410 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc01f110 con 0x55f34da32b60 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55b9ffb640 1 -- 172.21.15.31:0/3980278410 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f55cc01f920 con 0x55f34da32b60 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55da7ef740 1 -- 172.21.15.31:0/3980278410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55cc01fe00 con 0x55f34da32b60 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55b9ffb640 1 -- 172.21.15.31:0/3980278410 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f55cc03f050 con 0x55f34da32b60 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55da7ef740 10 client.25217 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55da7ef740 1 -- 172.21.15.31:0/3980278410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f34dada120 con 0x55f34da32b60 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55b9ffb640 1 -- 172.21.15.31:0/3980278410 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55cc01fe00 con 0x55f34da32b60 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55b9ffb640 1 -- 172.21.15.31:0/3980278410 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f55cc004a30 con 0x55f34da32b60 2024-09-16T00:42:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55da7ef740 10 client.25217 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55da7ef740 10 client.25217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55da7ef740 20 client.25217 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55da7ef740 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f34dada7d0 0x55f34dadcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.524+0000 7f55da7ef740 4 client.25217 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:42:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.525+0000 7f55da7ef740 1 -- 172.21.15.31:0/3980278410 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f34d99fd00 con 0x55f34dada7d0 2024-09-16T00:42:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.525+0000 7f55d59b1640 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f34dada7d0 0x55f34dadcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.525+0000 7f55d59b1640 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f34dada7d0 0x55f34dadcbf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f17f44cdf1ddb0 server_cookie=b2ab6325d949df72 in_seq=0 out_seq=0 2024-09-16T00:42:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.525+0000 7f55b9ffb640 10 client.25217 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:44.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.551+0000 7f55b9ffb640 1 -- 172.21.15.31:0/3980278410 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f34d99fd00 con 0x55f34dada7d0 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.551+0000 7f55b9ffb640 10 client.25217 handle_command_reply: tid=0 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.551+0000 7f55da7ef740 1 client.25217 shutdown 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 -- 172.21.15.31:0/3980278410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 msgr2=0x55f34da52a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da52a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55cc000c00 tx=0x7f55cc00b690 comp rx=0 tx=0).stop 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 -- 172.21.15.31:0/3980278410 shutdown_connections 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f34dada7d0 0x55f34dadcbf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34da3dd40 0x55f34da534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55b9ffb640 0 client.25217 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f34da33480 0x55f34da52fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 --2- 172.21.15.31:0/3980278410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f34da32b60 0x55f34da52a70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 -- 172.21.15.31:0/3980278410 >> 172.21.15.31:0/3980278410 conn(0x55f34da2f0f0 msgr2=0x55f34da2e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 -- 172.21.15.31:0/3980278410 shutdown_connections 2024-09-16T00:42:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 1 -- 172.21.15.31:0/3980278410 wait complete. 2024-09-16T00:42:44.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.552+0000 7f55da7ef740 20 client.25217 trim_cache size 0 max 16384 2024-09-16T00:42:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:42:44.554 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:42:44.554 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "a900d51d-ebcb-4e9d-8b8c-e45874ed2066", 2024-09-16T00:42:44.554 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:42:44.554 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:42:44.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 -- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55bc0a31e0 msgr2=0x7f55bc0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:44.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55bc0a31e0 0x7f55bc0a56a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55f34da2ff50 tx=0x7f55cc0046c0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 -- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55d0102790 msgr2=0x7f55d010d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:44.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55d0102790 0x7f55d010d280 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009a20 tx=0x7f55c00052b0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 -- 172.21.15.31:0/2168122832 shutdown_connections 2024-09-16T00:42:44.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55d0106f10 0x7f55d010d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55d0102790 0x7f55d010d280 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55bc0a31e0 0x7f55bc0a56a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 --2- 172.21.15.31:0/2168122832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0101de0 0x7f55d010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:44.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 -- 172.21.15.31:0/2168122832 >> 172.21.15.31:0/2168122832 conn(0x7f55d00fd8c0 msgr2=0x7f55d00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:44.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 -- 172.21.15.31:0/2168122832 shutdown_connections 2024-09-16T00:42:44.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:44.553+0000 7f55d8416640 1 -- 172.21.15.31:0/2168122832 wait complete. 2024-09-16T00:42:45.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:45 smithi031 ceph-mon[30956]: pgmap v404: 129 pgs: 129 active+clean; 34 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-09-16T00:42:45.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:45 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:42:45.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:45 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:45.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:45 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:42:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:45 smithi102 ceph-mon[35364]: pgmap v404: 129 pgs: 129 active+clean; 34 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-09-16T00:42:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:45 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:42:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:45 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:45 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:42:45.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:42:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:45 smithi081 ceph-mon[35539]: pgmap v404: 129 pgs: 129 active+clean; 34 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-09-16T00:42:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:45 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:42:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:45 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:45 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:42:45.864 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:42:46.186 INFO:tasks.workunit.client.0.smithi031.stdout: 16 261673 720 167340 907 98411 1289 2024-09-16T00:42:46.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.438+0000 7f81d6d88640 1 -- 172.21.15.31:0/2820919190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81d0112ba0 msgr2=0x7f81d0114f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.438+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2820919190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81d0112ba0 0x7f81d0114f90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009ca0 tx=0x7f81c802a950 comp rx=0 tx=0).stop 2024-09-16T00:42:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.438+0000 7f81d6d88640 1 -- 172.21.15.31:0/2820919190 shutdown_connections 2024-09-16T00:42:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.438+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2820919190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81d0112ba0 0x7f81d0114f90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.438+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2820919190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d0074eb0 0x7f81d0112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.438+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2820919190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d0074590 0x7f81d0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.438+0000 7f81d6d88640 1 -- 172.21.15.31:0/2820919190 >> 172.21.15.31:0/2820919190 conn(0x7f81d00fd8b0 msgr2=0x7f81d00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.438+0000 7f81d6d88640 1 -- 172.21.15.31:0/2820919190 shutdown_connections 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.438+0000 7f81d6d88640 1 -- 172.21.15.31:0/2820919190 wait complete. 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.439+0000 7f81d6d88640 1 Processor -- start 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.439+0000 7f81d6d88640 1 -- start start 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.439+0000 7f81d6d88640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d0074590 0x7f81d01a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.439+0000 7f81d6d88640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d0074eb0 0x7f81d01a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.439+0000 7f81d4b24640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d0074590 0x7f81d01a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.439+0000 7f81d6d88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81d0112ba0 0x7f81d019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.439+0000 7f81d4b24640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d0074590 0x7f81d01a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:35408/0 (socket says 172.21.15.31:35408) 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d4b24640 1 -- 172.21.15.31:0/2532497804 learned_addr learned my addr 172.21.15.31:0/2532497804 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d5325640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81d0112ba0 0x7f81d019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81cffff640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d0074eb0 0x7f81d01a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01185b0 con 0x7f81d0112ba0 2024-09-16T00:42:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f81d0118430 con 0x7f81d0074590 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81d0118730 con 0x7f81d0074eb0 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d4b24640 1 -- 172.21.15.31:0/2532497804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d0074eb0 msgr2=0x7f81d01a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d4b24640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d0074eb0 0x7f81d01a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d4b24640 1 -- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81d0112ba0 msgr2=0x7f81d019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d4b24640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81d0112ba0 0x7f81d019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d4b24640 1 -- 172.21.15.31:0/2532497804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d019f030 con 0x7f81d0074590 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.440+0000 7f81d4b24640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d0074590 0x7f81d01a4740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81c400e9f0 tx=0x7f81c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.441+0000 7f81cdffb640 1 -- 172.21.15.31:0/2532497804 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c400cd80 con 0x7f81d0074590 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.441+0000 7f81cdffb640 1 -- 172.21.15.31:0/2532497804 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c4010910 con 0x7f81d0074590 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.441+0000 7f81cdffb640 1 -- 172.21.15.31:0/2532497804 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4018990 con 0x7f81d0074590 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.441+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d019f320 con 0x7f81d0074590 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.441+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d01ab610 con 0x7f81d0074590 2024-09-16T00:42:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.444+0000 7f81cdffb640 1 -- 172.21.15.31:0/2532497804 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f81c4010ac0 con 0x7f81d0074590 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81d9161740 1 Processor -- start 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81cdffb640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f81c00a32c0 0x7f81c00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81d9161740 1 -- start start 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81d9161740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015a0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81cdffb640 1 -- 172.21.15.31:0/2532497804 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f81c40cff30 con 0x7f81d0074590 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81d9161740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81d9161740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81d5325640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f81c00a32c0 0x7f81c00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.446+0000 7f81cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36774/0 (socket says 172.21.15.31:36774) 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.447+0000 7f81d4b24640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.447+0000 7f81d4b24640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:48038/0 (socket says 172.21.15.31:48038) 2024-09-16T00:42:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.447+0000 7f81d9161740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a1015062e0 con 0x55a1015a0dc0 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.447+0000 7f81d9161740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a1014adce0 con 0x55a1015ab680 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.447+0000 7f81d9161740 1 -- 172.21.15.31:0/2930654835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a1015022a0 con 0x55a1015a04a0 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.447+0000 7f81cffff640 1 -- 172.21.15.31:0/2930654835 learned_addr learned my addr 172.21.15.31:0/2930654835 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.447+0000 7f81d5325640 1 --2- 172.21.15.31:0/2930654835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.448+0000 7f81d5325640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f81c00a32c0 0x7f81c00a5780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f81c8004af0 tx=0x7f81c802f3a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.448+0000 7f81d5325640 1 -- 172.21.15.31:0/2930654835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 msgr2=0x55a1015a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.448+0000 7f81d5325640 1 --2- 172.21.15.31:0/2930654835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.448+0000 7f81d5325640 1 -- 172.21.15.31:0/2930654835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 msgr2=0x55a1015ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.448+0000 7f81d5325640 1 --2- 172.21.15.31:0/2930654835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.448+0000 7f81d5325640 1 -- 172.21.15.31:0/2930654835 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a101492440 con 0x55a1015ab680 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.448+0000 7f81d5325640 1 --2- 172.21.15.31:0/2930654835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015ada70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81c802af40 tx=0x7f81c8047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81bbf06aac5c4ee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.450+0000 7f81baffd640 1 -- 172.21.15.31:0/2930654835 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8034040 con 0x55a1015ab680 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.450+0000 7f81baffd640 1 -- 172.21.15.31:0/2930654835 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c8003ec0 con 0x55a1015ab680 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.451+0000 7f81d9161740 1 -- 172.21.15.31:0/2930654835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 msgr2=0x55a1015ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.451+0000 7f81d9161740 1 --2- 172.21.15.31:0/2930654835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015ada70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81c802af40 tx=0x7f81c8047000 comp rx=0 tx=0).stop 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.452+0000 7f81d9161740 1 -- 172.21.15.31:0/2930654835 shutdown_connections 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.452+0000 7f81d9161740 1 --2- 172.21.15.31:0/2930654835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015ada70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.452+0000 7f81d9161740 1 --2- 172.21.15.31:0/2930654835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.452+0000 7f81d9161740 1 --2- 172.21.15.31:0/2930654835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.452+0000 7f81d9161740 1 -- 172.21.15.31:0/2930654835 >> 172.21.15.31:0/2930654835 conn(0x55a10159ca30 msgr2=0x55a10159ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.453+0000 7f81d9161740 1 -- 172.21.15.31:0/2930654835 shutdown_connections 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.453+0000 7f81d9161740 1 -- 172.21.15.31:0/2930654835 wait complete. 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.453+0000 7f81d9161740 1 -- start start 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.454+0000 7f81d9161740 1 Processor -- start 2024-09-16T00:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.454+0000 7f81d9161740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.454+0000 7f81d9161740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015a8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.454+0000 7f81d9161740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.454+0000 7f81d9161740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a1015ae0d0 con 0x55a1015a0dc0 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.454+0000 7f81d9161740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a1014c1050 con 0x55a1015ab680 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.454+0000 7f81d9161740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a1015ae250 con 0x55a1015a04a0 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.454+0000 7f81d9161740 10 client.0 _ll_register_callbacks cb 0x55a101501160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.455+0000 7f81cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.455+0000 7f81cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015a8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36780/0 (socket says 172.21.15.31:36780) 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.455+0000 7f81cffff640 1 -- 172.21.15.31:0/3522712530 learned_addr learned my addr 172.21.15.31:0/3522712530 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.455+0000 7f81cffff640 1 -- 172.21.15.31:0/3522712530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 msgr2=0x55a1015aaf70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.455+0000 7f81cffff640 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015aaf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.455+0000 7f81cffff640 1 -- 172.21.15.31:0/3522712530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 msgr2=0x55a1015a8760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:42:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.455+0000 7f81cffff640 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015a8760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.455+0000 7f81cffff640 1 -- 172.21.15.31:0/3522712530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1012773c0 con 0x55a1015a0dc0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.456+0000 7f81cffff640 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015a8220 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81bc012a70 tx=0x7f81bc012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e67d9604e428e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.456+0000 7f81affff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.456+0000 7f81affff640 1 -- 172.21.15.31:0/3522712530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0120d0 con 0x55a1015a0dc0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.457+0000 7f81affff640 1 -- 172.21.15.31:0/3522712530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f81bc00f980 con 0x55a1015a0dc0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.457+0000 7f81d9161740 1 -- 172.21.15.31:0/3522712530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c8003ec0 con 0x55a1015a0dc0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.458+0000 7f81affff640 1 -- 172.21.15.31:0/3522712530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f81bc012270 con 0x55a1015a0dc0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.458+0000 7f81d9161740 10 client.15537 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.458+0000 7f81d9161740 1 -- 172.21.15.31:0/3522712530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a1015b9b40 con 0x55a1015a0dc0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.459+0000 7f81affff640 1 -- 172.21.15.31:0/3522712530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c8003ec0 con 0x55a1015a0dc0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.459+0000 7f81affff640 1 -- 172.21.15.31:0/3522712530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f81bc010430 con 0x55a1015a0dc0 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.459+0000 7f81d9161740 10 client.15537 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.459+0000 7f81d9161740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.459+0000 7f81d9161740 20 client.15537 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.459+0000 7f81d9161740 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a1015baf50 0x55a1015bd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.459+0000 7f81d9161740 4 client.15537 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:42:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.459+0000 7f81d9161740 1 -- 172.21.15.31:0/3522712530 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a101513850 con 0x55a1015baf50 2024-09-16T00:42:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.460+0000 7f81d4b24640 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a1015baf50 0x55a1015bd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.460+0000 7f81affff640 10 client.15537 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.460+0000 7f81d4b24640 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a1015baf50 0x55a1015bd3b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c5c371ce2dd2ab8 server_cookie=83ee334307186d3d in_seq=0 out_seq=0 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.463+0000 7f81affff640 1 -- 172.21.15.31:0/3522712530 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a101513850 con 0x55a1015baf50 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.463+0000 7f81affff640 10 client.15537 handle_command_reply: tid=0 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.463+0000 7f81d9161740 1 client.15537 shutdown 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.463+0000 7f81d9161740 1 -- 172.21.15.31:0/3522712530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 msgr2=0x55a1015a8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.463+0000 7f81d9161740 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015a8220 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81bc012a70 tx=0x7f81bc012f30 comp rx=0 tx=0).stop 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81d9161740 1 -- 172.21.15.31:0/3522712530 shutdown_connections 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81d9161740 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81d9161740 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a0dc0 0x55a1015a8220 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81d9161740 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a1015baf50 0x55a1015bd3b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81d9161740 1 --2- 172.21.15.31:0/3522712530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81d9161740 1 -- 172.21.15.31:0/3522712530 >> 172.21.15.31:0/3522712530 conn(0x55a10159ca30 msgr2=0x55a1015a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81affff640 0 client.15537 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81d9161740 1 -- 172.21.15.31:0/3522712530 shutdown_connections 2024-09-16T00:42:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81d9161740 1 -- 172.21.15.31:0/3522712530 wait complete. 2024-09-16T00:42:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.464+0000 7f81d9161740 20 client.15537 trim_cache size 0 max 16384 2024-09-16T00:42:46.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.490+0000 7f81d9161740 1 Processor -- start 2024-09-16T00:42:46.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.490+0000 7f81d9161740 1 -- start start 2024-09-16T00:42:46.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d9161740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015a04a0 0x55a1015b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d9161740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015ab680 0x55a1015c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d9161740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d9161740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01b34b0 con 0x55a1015a3b60 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d9161740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01b5d30 con 0x55a1015a04a0 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d9161740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01b3b20 con 0x55a1015ab680 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d4b24640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015a04a0 0x55a1015b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d4b24640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015a04a0 0x55a1015b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:35418/0 (socket says 172.21.15.31:35418) 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d4b24640 1 -- 172.21.15.31:0/2547534249 learned_addr learned my addr 172.21.15.31:0/2547534249 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81cffff640 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015ab680 0x55a1015c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d5325640 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d4b24640 1 -- 172.21.15.31:0/2547534249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015ab680 msgr2=0x55a1015c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:46.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d4b24640 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015ab680 0x55a1015c2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d4b24640 1 -- 172.21.15.31:0/2547534249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 msgr2=0x55a1015c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:46.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d4b24640 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015c31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.491+0000 7f81d4b24640 1 -- 172.21.15.31:0/2547534249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10159dbc0 con 0x55a1015a04a0 2024-09-16T00:42:46.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.492+0000 7f81cffff640 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015ab680 0x55a1015c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:46.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.492+0000 7f81d4b24640 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015a04a0 0x55a1015b73d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81c40986b0 tx=0x7f81c4018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ee947584efc0fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:46.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.492+0000 7f81ba7fc640 1 -- 172.21.15.31:0/2547534249 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40b0070 con 0x55a1015a04a0 2024-09-16T00:42:46.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.492+0000 7f81ba7fc640 1 -- 172.21.15.31:0/2547534249 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c409a2f0 con 0x55a1015a04a0 2024-09-16T00:42:46.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.492+0000 7f81ba7fc640 1 -- 172.21.15.31:0/2547534249 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40a00d0 con 0x55a1015a04a0 2024-09-16T00:42:46.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.493+0000 7f81d9161740 1 -- 172.21.15.31:0/2547534249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015a04a0 msgr2=0x55a1015b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:46.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.493+0000 7f81d9161740 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015a04a0 0x55a1015b73d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81c40986b0 tx=0x7f81c4018420 comp rx=0 tx=0).stop 2024-09-16T00:42:46.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.493+0000 7f81d9161740 1 -- 172.21.15.31:0/2547534249 shutdown_connections 2024-09-16T00:42:46.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.493+0000 7f81d9161740 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.493+0000 7f81d9161740 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015ab680 0x55a1015c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.493+0000 7f81d9161740 1 --2- 172.21.15.31:0/2547534249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015a04a0 0x55a1015b73d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.493+0000 7f81d9161740 1 -- 172.21.15.31:0/2547534249 >> 172.21.15.31:0/2547534249 conn(0x55a10159ca30 msgr2=0x55a1015aa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:46.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.493+0000 7f81d9161740 1 -- 172.21.15.31:0/2547534249 shutdown_connections 2024-09-16T00:42:46.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.493+0000 7f81d9161740 1 -- 172.21.15.31:0/2547534249 wait complete. 2024-09-16T00:42:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.494+0000 7f81d9161740 1 -- start start 2024-09-16T00:42:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.494+0000 7f81d9161740 1 Processor -- start 2024-09-16T00:42:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81d9161740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81d9161740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81d4b24640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81d4b24640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:48068/0 (socket says 172.21.15.31:48068) 2024-09-16T00:42:46.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36794/0 (socket says 172.21.15.31:36794) 2024-09-16T00:42:46.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81d4b24640 1 -- 172.21.15.31:0/3188787893 learned_addr learned my addr 172.21.15.31:0/3188787893 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:42:46.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81d9161740 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01b41f0 con 0x55a1015a3b60 2024-09-16T00:42:46.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.495+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01b5660 con 0x55a1015ab680 2024-09-16T00:42:46.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01b48c0 con 0x55a1015a04a0 2024-09-16T00:42:46.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d5325640 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d5325640 1 -- 172.21.15.31:0/3188787893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 msgr2=0x55a1015b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:46.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d5325640 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d5325640 1 -- 172.21.15.31:0/3188787893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 msgr2=0x55a1015b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:42:46.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d5325640 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d5325640 1 -- 172.21.15.31:0/3188787893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1015b0ad0 con 0x55a1015ab680 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81cffff640 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d9161740 10 client.0 _ll_register_callbacks cb 0x7f81bc009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c409a2f0 con 0x55a1015ab680 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81d5325640 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015b0510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55a1015a3fe0 tx=0x7f81c8030000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=478cea4ecaa3362e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.496+0000 7f81affff640 10 client.25232 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81affff640 1 -- 172.21.15.31:0/3188787893 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8034040 con 0x55a1015ab680 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81affff640 1 -- 172.21.15.31:0/3188787893 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f81c8048a50 con 0x55a1015ab680 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81affff640 1 -- 172.21.15.31:0/3188787893 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f81c8048c70 con 0x55a1015ab680 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81d9161740 10 client.25232 fetch_fsmap learned FSMap version 20 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=20}) -- 0x55a101626220 con 0x55a1015ab680 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81affff640 1 -- 172.21.15.31:0/3188787893 <== mon.2 v2:172.21.15.81:3300/0 4 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f81c804f020 con 0x55a1015ab680 2024-09-16T00:42:46.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81d9161740 10 client.25232 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:42:46.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81d9161740 10 client.25232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:42:46.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81d9161740 20 client.25232 populate_metadata read hostname 'smithi031' 2024-09-16T00:42:46.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81d9161740 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a101626550 0x55a101628a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:46.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81d9161740 4 client.25232 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:42:46.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.497+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a10150dd60 con 0x55a101626550 2024-09-16T00:42:46.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.498+0000 7f81d4b24640 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a101626550 0x55a101628a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:46.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.498+0000 7f81affff640 10 client.25232 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:46.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.498+0000 7f81d4b24640 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a101626550 0x55a101628a10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e886c28e2c86a7b2 server_cookie=1a724bec21e454a5 in_seq=0 out_seq=0 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81affff640 1 -- 172.21.15.31:0/3188787893 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55a10150dd60 con 0x55a101626550 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81affff640 10 client.25232 handle_command_reply: tid=0 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 client.25232 shutdown 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 msgr2=0x55a1015b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015b0510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55a1015a3fe0 tx=0x7f81c8030000 comp rx=0 tx=0).stop 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 shutdown_connections 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a1015ab680 0x55a1015b0510 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a1015a3b60 0x55a1015b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a101626550 0x55a101628a10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81affff640 0 client.25232 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 --2- 172.21.15.31:0/3188787893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1015a04a0 0x55a1015b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 >> 172.21.15.31:0/3188787893 conn(0x55a10159ca30 msgr2=0x55a1015a1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 shutdown_connections 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 1 -- 172.21.15.31:0/3188787893 wait complete. 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.503+0000 7f81d9161740 20 client.25232 trim_cache size 0 max 16384 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (1784 inodes in the stack)", 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: "a900d51d-ebcb-4e9d-8b8c-e45874ed2066": { 2024-09-16T00:42:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:42:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "a900d51d-ebcb-4e9d-8b8c-e45874ed2066", 2024-09-16T00:42:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:42:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:42:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:42:46.506 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f81c00a32c0 msgr2=0x7f81c00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f81c00a32c0 0x7f81c00a5780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f81c8004af0 tx=0x7f81c802f3a0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d0074590 msgr2=0x7f81d01a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d0074590 0x7f81d01a4740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81c400e9f0 tx=0x7f81c400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 shutdown_connections 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81d0112ba0 0x7f81d019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f81c00a32c0 0x7f81c00a5780 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d0074eb0 0x7f81d01a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 --2- 172.21.15.31:0/2532497804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d0074590 0x7f81d01a4740 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.504+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 >> 172.21.15.31:0/2532497804 conn(0x7f81d00fd8b0 msgr2=0x7f81d00ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:46.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.505+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 shutdown_connections 2024-09-16T00:42:46.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:42:46.505+0000 7f81d6d88640 1 -- 172.21.15.31:0/2532497804 wait complete. 2024-09-16T00:42:47.246 INFO:tasks.cephfs.filesystem:scrub status for tag:a900d51d-ebcb-4e9d-8b8c-e45874ed2066 - {'path': '/', 'tag': 'a900d51d-ebcb-4e9d-8b8c-e45874ed2066', 'options': 'recursive,force'} 2024-09-16T00:42:47.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:47 smithi102 ceph-mon[35364]: pgmap v405: 129 pgs: 129 active+clean; 52 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 208 op/s 2024-09-16T00:42:47.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:47 smithi031 ceph-mon[30956]: pgmap v405: 129 pgs: 129 active+clean; 52 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 208 op/s 2024-09-16T00:42:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:47 smithi081 ceph-mon[35539]: pgmap v405: 129 pgs: 129 active+clean; 52 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 208 op/s 2024-09-16T00:42:49.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:49 smithi102 ceph-mon[35364]: pgmap v406: 129 pgs: 129 active+clean; 59 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 235 op/s 2024-09-16T00:42:49.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:49 smithi031 ceph-mon[30956]: pgmap v406: 129 pgs: 129 active+clean; 59 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 235 op/s 2024-09-16T00:42:49.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:49 smithi081 ceph-mon[35539]: pgmap v406: 129 pgs: 129 active+clean; 59 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 235 op/s 2024-09-16T00:42:51.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:51 smithi102 ceph-mon[35364]: pgmap v407: 129 pgs: 129 active+clean; 60 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 229 op/s 2024-09-16T00:42:51.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:51 smithi031 ceph-mon[30956]: pgmap v407: 129 pgs: 129 active+clean; 60 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 229 op/s 2024-09-16T00:42:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:51 smithi081 ceph-mon[35539]: pgmap v407: 129 pgs: 129 active+clean; 60 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 229 op/s 2024-09-16T00:42:53.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:53 smithi102 ceph-mon[35364]: pgmap v408: 129 pgs: 129 active+clean; 62 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 227 op/s 2024-09-16T00:42:53.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:53 smithi031 ceph-mon[30956]: pgmap v408: 129 pgs: 129 active+clean; 62 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 227 op/s 2024-09-16T00:42:53.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:53 smithi081 ceph-mon[35539]: pgmap v408: 129 pgs: 129 active+clean; 62 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 227 op/s 2024-09-16T00:42:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:55 smithi102 ceph-mon[35364]: pgmap v409: 129 pgs: 129 active+clean; 68 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 234 op/s 2024-09-16T00:42:55.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:55 smithi031 ceph-mon[30956]: pgmap v409: 129 pgs: 129 active+clean; 68 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 234 op/s 2024-09-16T00:42:55.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:55 smithi081 ceph-mon[35539]: pgmap v409: 129 pgs: 129 active+clean; 68 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 234 op/s 2024-09-16T00:42:56.186 INFO:tasks.workunit.client.0.smithi031.stdout: 30 753620 753 461005 584 425412 1751 2024-09-16T00:42:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:57 smithi102 ceph-mon[35364]: pgmap v410: 129 pgs: 129 active+clean; 69 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 144 op/s 2024-09-16T00:42:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:42:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:57 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:57 smithi031 ceph-mon[30956]: pgmap v410: 129 pgs: 129 active+clean; 69 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 144 op/s 2024-09-16T00:42:57.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:42:57.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:57 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:57 smithi081 ceph-mon[35539]: pgmap v410: 129 pgs: 129 active+clean; 69 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 144 op/s 2024-09-16T00:42:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:42:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:57 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:58.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:42:58.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:42:58.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:58.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:58 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:58.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:58 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:42:58.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:42:58.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:42:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:58 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:58.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:58 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:42:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:42:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:42:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:42:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:58 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:42:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:58 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:42:59.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:42:59 smithi102 ceph-mon[35364]: pgmap v411: 129 pgs: 129 active+clean; 73 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-09-16T00:42:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:42:59 smithi031 ceph-mon[30956]: pgmap v411: 129 pgs: 129 active+clean; 73 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-09-16T00:42:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:42:59 smithi081 ceph-mon[35539]: pgmap v411: 129 pgs: 129 active+clean; 73 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-09-16T00:43:01.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:01 smithi102 ceph-mon[35364]: pgmap v412: 129 pgs: 129 active+clean; 76 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 62 op/s 2024-09-16T00:43:01.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:01 smithi031 ceph-mon[30956]: pgmap v412: 129 pgs: 129 active+clean; 76 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 62 op/s 2024-09-16T00:43:01.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:01 smithi081 ceph-mon[35539]: pgmap v412: 129 pgs: 129 active+clean; 76 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 62 op/s 2024-09-16T00:43:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:03 smithi102 ceph-mon[35364]: pgmap v413: 129 pgs: 129 active+clean; 83 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 55 op/s 2024-09-16T00:43:03.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:03 smithi031 ceph-mon[30956]: pgmap v413: 129 pgs: 129 active+clean; 83 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 55 op/s 2024-09-16T00:43:03.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:03 smithi081 ceph-mon[35539]: pgmap v413: 129 pgs: 129 active+clean; 83 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 55 op/s 2024-09-16T00:43:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:05 smithi102 ceph-mon[35364]: pgmap v414: 129 pgs: 129 active+clean; 105 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 195 op/s 2024-09-16T00:43:05.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:05 smithi031 ceph-mon[30956]: pgmap v414: 129 pgs: 129 active+clean; 105 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 195 op/s 2024-09-16T00:43:05.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:05 smithi081 ceph-mon[35539]: pgmap v414: 129 pgs: 129 active+clean; 105 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 195 op/s 2024-09-16T00:43:06.189 INFO:tasks.workunit.client.0.smithi031.stdout: 36 692408 486 421883 406 392107 988 2024-09-16T00:43:07.548 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:07 smithi081 ceph-mon[35539]: pgmap v415: 129 pgs: 129 active+clean; 122 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-09-16T00:43:07.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:07 smithi102 ceph-mon[35364]: pgmap v415: 129 pgs: 129 active+clean; 122 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-09-16T00:43:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:07 smithi031 ceph-mon[30956]: pgmap v415: 129 pgs: 129 active+clean; 122 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-09-16T00:43:09.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:09 smithi102 ceph-mon[35364]: pgmap v416: 129 pgs: 129 active+clean; 152 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.7 MiB/s wr, 449 op/s 2024-09-16T00:43:09.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:09 smithi031 ceph-mon[30956]: pgmap v416: 129 pgs: 129 active+clean; 152 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.7 MiB/s wr, 449 op/s 2024-09-16T00:43:09.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:09 smithi081 ceph-mon[35539]: pgmap v416: 129 pgs: 129 active+clean; 152 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.7 MiB/s wr, 449 op/s 2024-09-16T00:43:11.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:11 smithi102 ceph-mon[35364]: pgmap v417: 129 pgs: 129 active+clean; 161 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.3 MiB/s wr, 521 op/s 2024-09-16T00:43:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:11 smithi031 ceph-mon[30956]: pgmap v417: 129 pgs: 129 active+clean; 161 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.3 MiB/s wr, 521 op/s 2024-09-16T00:43:11.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:11 smithi081 ceph-mon[35539]: pgmap v417: 129 pgs: 129 active+clean; 161 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.3 MiB/s wr, 521 op/s 2024-09-16T00:43:13.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:13 smithi031 ceph-mon[30956]: pgmap v418: 129 pgs: 129 active+clean; 170 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.0 MiB/s wr, 618 op/s 2024-09-16T00:43:13.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:13 smithi081 ceph-mon[35539]: pgmap v418: 129 pgs: 129 active+clean; 170 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.0 MiB/s wr, 618 op/s 2024-09-16T00:43:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:13 smithi102 ceph-mon[35364]: pgmap v418: 129 pgs: 129 active+clean; 170 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.0 MiB/s wr, 618 op/s 2024-09-16T00:43:14.038 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:43:14.038 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-16T00:43:14.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:43:14.540 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.136+0000 7fa04a381640 1 -- 172.21.15.31:0/1414756854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102ef0 msgr2=0x7fa04410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.136+0000 7fa04a381640 1 --2- 172.21.15.31:0/1414756854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102ef0 0x7fa04410da40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009ca0 tx=0x7fa03c02a950 comp rx=0 tx=0).stop 2024-09-16T00:43:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.136+0000 7fa04a381640 1 -- 172.21.15.31:0/1414756854 shutdown_connections 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.136+0000 7fa04a381640 1 --2- 172.21.15.31:0/1414756854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102ef0 0x7fa04410da40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.136+0000 7fa04a381640 1 --2- 172.21.15.31:0/1414756854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa044102570 0x7fa0441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.136+0000 7fa04a381640 1 --2- 172.21.15.31:0/1414756854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa044100e20 0x7fa044101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.136+0000 7fa04a381640 1 -- 172.21.15.31:0/1414756854 >> 172.21.15.31:0/1414756854 conn(0x7fa0440fc7d0 msgr2=0x7fa0440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.137+0000 7fa04a381640 1 -- 172.21.15.31:0/1414756854 shutdown_connections 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.137+0000 7fa04a381640 1 -- 172.21.15.31:0/1414756854 wait complete. 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.137+0000 7fa04a381640 1 Processor -- start 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.137+0000 7fa04a381640 1 -- start start 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.137+0000 7fa04a381640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa044100e20 0x7fa0441992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.137+0000 7fa04a381640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102570 0x7fa0441997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:15.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa04a381640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa044102ef0 0x7fa0441a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa04a381640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa044110f80 con 0x7fa044102570 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa04a381640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa044110e00 con 0x7fa044100e20 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa04a381640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa044111100 con 0x7fa044102ef0 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102570 0x7fa0441997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa03bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102570 0x7fa0441997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49660/0 (socket says 172.21.15.31:49660) 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa03bfff640 1 -- 172.21.15.31:0/3637523088 learned_addr learned my addr 172.21.15.31:0/3637523088 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa043fff640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa044100e20 0x7fa0441992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa04891e640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa044102ef0 0x7fa0441a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa03bfff640 1 -- 172.21.15.31:0/3637523088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa044102ef0 msgr2=0x7fa0441a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa03bfff640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa044102ef0 0x7fa0441a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa03bfff640 1 -- 172.21.15.31:0/3637523088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa044100e20 msgr2=0x7fa0441992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa03bfff640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa044100e20 0x7fa0441992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa03bfff640 1 -- 172.21.15.31:0/3637523088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0441a4870 con 0x7fa044102570 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa03bfff640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102570 0x7fa0441997f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00e9f0 tx=0x7fa02c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.138+0000 7fa041ffb640 1 -- 172.21.15.31:0/3637523088 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c00cd80 con 0x7fa044102570 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.139+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0441a4a40 con 0x7fa044102570 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.139+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0441a4f80 con 0x7fa044102570 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.139+0000 7fa041ffb640 1 -- 172.21.15.31:0/3637523088 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa02c010910 con 0x7fa044102570 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.139+0000 7fa041ffb640 1 -- 172.21.15.31:0/3637523088 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c018980 con 0x7fa044102570 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.140+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa044101200 con 0x7fa044102570 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.141+0000 7fa041ffb640 1 -- 172.21.15.31:0/3637523088 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa02c010ac0 con 0x7fa044102570 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.142+0000 7fa041ffb640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0300a3240 0x7fa0300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.142+0000 7fa041ffb640 1 -- 172.21.15.31:0/3637523088 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa02c0cfff0 con 0x7fa044102570 2024-09-16T00:43:15.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.142+0000 7fa043fff640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0300a3240 0x7fa0300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:15.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.142+0000 7fa043fff640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0300a3240 0x7fa0300a5700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa03400bbe0 tx=0x7fa03400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:15.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.146+0000 7fa041ffb640 1 -- 172.21.15.31:0/3637523088 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa02c09c310 con 0x7fa044102570 2024-09-16T00:43:15.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.392+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0441a5210 con 0x7fa044102570 2024-09-16T00:43:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.393+0000 7fa041ffb640 1 -- 172.21.15.31:0/3637523088 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fa02c0a0610 con 0x7fa044102570 2024-09-16T00:43:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:43:15.393 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:43:15.393 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:43:15.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0300a3240 msgr2=0x7fa0300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:15.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0300a3240 0x7fa0300a5700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa03400bbe0 tx=0x7fa03400b390 comp rx=0 tx=0).stop 2024-09-16T00:43:15.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102570 msgr2=0x7fa0441997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102570 0x7fa0441997f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00e9f0 tx=0x7fa02c00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 shutdown_connections 2024-09-16T00:43:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0300a3240 0x7fa0300a5700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa044102ef0 0x7fa0441a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044102570 0x7fa0441997f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 --2- 172.21.15.31:0/3637523088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa044100e20 0x7fa0441992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 >> 172.21.15.31:0/3637523088 conn(0x7fa0440fc7d0 msgr2=0x7fa044106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 shutdown_connections 2024-09-16T00:43:15.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:15.395+0000 7fa04a381640 1 -- 172.21.15.31:0/3637523088 wait complete. 2024-09-16T00:43:15.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:15 smithi031 ceph-mon[30956]: pgmap v419: 129 pgs: 129 active+clean; 178 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.3 MiB/s wr, 650 op/s 2024-09-16T00:43:15.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:15 smithi081 ceph-mon[35539]: pgmap v419: 129 pgs: 129 active+clean; 178 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.3 MiB/s wr, 650 op/s 2024-09-16T00:43:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:15 smithi102 ceph-mon[35364]: pgmap v419: 129 pgs: 129 active+clean; 178 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.3 MiB/s wr, 650 op/s 2024-09-16T00:43:16.186 INFO:tasks.workunit.client.0.smithi031.stdout: 44 1227802 474 776982 325 659544 1383 2024-09-16T00:43:16.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:16 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3637523088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:16.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:16 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3637523088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:16.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:16 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3637523088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:17.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:43:17.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:17 smithi031 ceph-mon[30956]: pgmap v420: 129 pgs: 129 active+clean; 177 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 521 op/s 2024-09-16T00:43:17.576 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-16T00:43:17.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-16T00:43:17.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:17 smithi081 ceph-mon[35539]: pgmap v420: 129 pgs: 129 active+clean; 177 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 521 op/s 2024-09-16T00:43:17.759 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:17 smithi102 ceph-mon[35364]: pgmap v420: 129 pgs: 129 active+clean; 177 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 521 op/s 2024-09-16T00:43:18.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.562+0000 7fcbbb97f640 1 -- 172.21.15.31:0/2033815265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb4074990 msgr2=0x7fcbb407b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:18.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.562+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/2033815265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb4074990 0x7fcbb407b3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00b4a0 tx=0x7fcbac02ba00 comp rx=0 tx=0).stop 2024-09-16T00:43:18.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.563+0000 7fcbbb97f640 1 -- 172.21.15.31:0/2033815265 shutdown_connections 2024-09-16T00:43:18.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.563+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/2033815265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb4074990 0x7fcbb407b3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.563+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/2033815265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbb4109e00 0x7fcbb40742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.563+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/2033815265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb4109450 0x7fcbb4109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.563+0000 7fcbbb97f640 1 -- 172.21.15.31:0/2033815265 >> 172.21.15.31:0/2033815265 conn(0x7fcbb406e140 msgr2=0x7fcbb406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:18.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.565+0000 7fcbbb97f640 1 -- 172.21.15.31:0/2033815265 shutdown_connections 2024-09-16T00:43:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.567+0000 7fcbbb97f640 1 -- 172.21.15.31:0/2033815265 wait complete. 2024-09-16T00:43:18.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.568+0000 7fcbbb97f640 1 Processor -- start 2024-09-16T00:43:18.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.576+0000 7fcbbb97f640 1 -- start start 2024-09-16T00:43:18.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.576+0000 7fcbbb97f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb4074990 0x7fcbb407b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.576+0000 7fcbbb97f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbb4109450 0x7fcbb40761d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.576+0000 7fcbbb97f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb4109e00 0x7fcbb4076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.576+0000 7fcbbb97f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb407e7e0 con 0x7fcbb4109450 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.576+0000 7fcbbb97f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb407e660 con 0x7fcbb4109e00 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.576+0000 7fcbbb97f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb407e960 con 0x7fcbb4074990 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.577+0000 7fcbb9f1c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb4109e00 0x7fcbb4076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.577+0000 7fcbb8f1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbb4109450 0x7fcbb40761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.577+0000 7fcbb8f1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbb4109450 0x7fcbb40761d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39278/0 (socket says 172.21.15.31:39278) 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.577+0000 7fcbb8f1a640 1 -- 172.21.15.31:0/609548934 learned_addr learned my addr 172.21.15.31:0/609548934 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.578+0000 7fcbb8f1a640 1 -- 172.21.15.31:0/609548934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb4074990 msgr2=0x7fcbb407b140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.578+0000 7fcbb8f1a640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb4074990 0x7fcbb407b140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.578+0000 7fcbb8f1a640 1 -- 172.21.15.31:0/609548934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb4109e00 msgr2=0x7fcbb4076710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.578+0000 7fcbb8f1a640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb4109e00 0x7fcbb4076710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.578+0000 7fcbb8f1a640 1 -- 172.21.15.31:0/609548934 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb4076fa0 con 0x7fcbb4109450 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.579+0000 7fcbb8f1a640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbb4109450 0x7fcbb40761d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006fd0 tx=0x7fcba400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.579+0000 7fcbaa7fc640 1 -- 172.21.15.31:0/609548934 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4011830 con 0x7fcbb4109450 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.579+0000 7fcbaa7fc640 1 -- 172.21.15.31:0/609548934 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba4016070 con 0x7fcbb4109450 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.579+0000 7fcbbb97f640 1 -- 172.21.15.31:0/609548934 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb41bc000 con 0x7fcbb4109450 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.580+0000 7fcbbb97f640 1 -- 172.21.15.31:0/609548934 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb41bc540 con 0x7fcbb4109450 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbbdd58740 1 Processor -- start 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbaa7fc640 1 -- 172.21.15.31:0/609548934 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba400f2e0 con 0x7fcbb4109450 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbbdd58740 1 -- start start 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266b4a0 0x55bb5266b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 0x55bb52676140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 0x55bb52678a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbbdd58740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bb525d12e0 con 0x55bb52676680 2024-09-16T00:43:18.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbbdd58740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb52578ce0 con 0x55bb5266bdc0 2024-09-16T00:43:18.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbb971b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 0x55bb52676140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbb971b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 0x55bb52676140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38362/0 (socket says 172.21.15.31:38362) 2024-09-16T00:43:18.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbb971b640 1 -- 172.21.15.31:0/3635905116 learned_addr learned my addr 172.21.15.31:0/3635905116 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:18.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.584+0000 7fcbbdd58740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bb525cd2a0 con 0x55bb5266b4a0 2024-09-16T00:43:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.592+0000 7fcbb971b640 1 -- 172.21.15.31:0/3635905116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266b4a0 msgr2=0x55bb5266b880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:43:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.592+0000 7fcbb971b640 1 --2- 172.21.15.31:0/3635905116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266b4a0 0x55bb5266b880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.592+0000 7fcbb971b640 1 -- 172.21.15.31:0/3635905116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 msgr2=0x55bb52678a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.592+0000 7fcbb971b640 1 --2- 172.21.15.31:0/3635905116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 0x55bb52678a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.592+0000 7fcbb971b640 1 -- 172.21.15.31:0/3635905116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5255d440 con 0x55bb5266bdc0 2024-09-16T00:43:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.593+0000 7fcbaa7fc640 1 -- 172.21.15.31:0/609548934 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcba40119d0 con 0x7fcbb4109450 2024-09-16T00:43:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.593+0000 7fcbb971b640 1 --2- 172.21.15.31:0/3635905116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 0x55bb52676140 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009bd0 tx=0x7fcbb000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b209226fa345c68a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.594+0000 7fcb9f7fe640 1 -- 172.21.15.31:0/3635905116 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0005020 con 0x55bb5266bdc0 2024-09-16T00:43:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.597+0000 7fcb9f7fe640 1 -- 172.21.15.31:0/3635905116 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb00047a0 con 0x55bb5266bdc0 2024-09-16T00:43:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.597+0000 7fcb9f7fe640 1 -- 172.21.15.31:0/3635905116 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0010690 con 0x55bb5266bdc0 2024-09-16T00:43:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.602+0000 7fcbbdd58740 1 -- 172.21.15.31:0/3635905116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 msgr2=0x55bb52676140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.602+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/3635905116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 0x55bb52676140 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009bd0 tx=0x7fcbb000e720 comp rx=0 tx=0).stop 2024-09-16T00:43:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.602+0000 7fcbbdd58740 1 -- 172.21.15.31:0/3635905116 shutdown_connections 2024-09-16T00:43:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.602+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/3635905116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 0x55bb52678a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.602+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/3635905116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 0x55bb52676140 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.602+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/3635905116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266b4a0 0x55bb5266b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.602+0000 7fcbbdd58740 1 -- 172.21.15.31:0/3635905116 >> 172.21.15.31:0/3635905116 conn(0x55bb52667a30 msgr2=0x55bb52667e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.603+0000 7fcbaa7fc640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcba00a3460 0x7fcba00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.603+0000 7fcbaa7fc640 1 -- 172.21.15.31:0/609548934 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fcba40d13b0 con 0x7fcbb4109450 2024-09-16T00:43:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.604+0000 7fcbb971b640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcba00a3460 0x7fcba00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.605+0000 7fcbb971b640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcba00a3460 0x7fcba00a5920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009ba0 tx=0x7fcbb000ef50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:18.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.610+0000 7fcbbdd58740 1 -- 172.21.15.31:0/3635905116 shutdown_connections 2024-09-16T00:43:18.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.611+0000 7fcbbdd58740 1 -- 172.21.15.31:0/3635905116 wait complete. 2024-09-16T00:43:18.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.611+0000 7fcbbdd58740 1 -- start start 2024-09-16T00:43:18.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.611+0000 7fcbbdd58740 1 Processor -- start 2024-09-16T00:43:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.612+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb52675f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.612+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266bdc0 0x55bb52673240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.612+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb52676680 0x55bb52673780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.612+0000 7fcbbdd58740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bb52679250 con 0x55bb5266b4a0 2024-09-16T00:43:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.612+0000 7fcbbdd58740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb5258c050 con 0x55bb52676680 2024-09-16T00:43:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.612+0000 7fcbbdd58740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bb526793d0 con 0x55bb5266bdc0 2024-09-16T00:43:18.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.612+0000 7fcbbdd58740 10 client.0 _ll_register_callbacks cb 0x55bb525cc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.614+0000 7fcbb9f1c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb52676680 0x55bb52673780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.614+0000 7fcbb9f1c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb52676680 0x55bb52673780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38368/0 (socket says 172.21.15.31:38368) 2024-09-16T00:43:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.614+0000 7fcbb9f1c640 1 -- 172.21.15.31:0/813753918 learned_addr learned my addr 172.21.15.31:0/813753918 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:18.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.614+0000 7fcbb971b640 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb52675f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.614+0000 7fcbb971b640 1 -- 172.21.15.31:0/813753918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266bdc0 msgr2=0x55bb52673240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:43:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.614+0000 7fcbb971b640 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266bdc0 0x55bb52673240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.614+0000 7fcbb971b640 1 -- 172.21.15.31:0/813753918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb52676680 msgr2=0x55bb52673780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.614+0000 7fcbb971b640 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb52676680 0x55bb52673780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.614+0000 7fcbb971b640 1 -- 172.21.15.31:0/813753918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb523423c0 con 0x55bb5266b4a0 2024-09-16T00:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.615+0000 7fcbb971b640 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb52675f90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0000c00 tx=0x7fcbb0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7629cd7d8c3e603a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.615+0000 7fcb9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.615+0000 7fcb9cff9640 1 -- 172.21.15.31:0/813753918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0034060 con 0x55bb5266b4a0 2024-09-16T00:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.615+0000 7fcbbdd58740 1 -- 172.21.15.31:0/813753918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb00047a0 con 0x55bb5266b4a0 2024-09-16T00:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.615+0000 7fcb9cff9640 1 -- 172.21.15.31:0/813753918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fcbb0010bc0 con 0x55bb5266b4a0 2024-09-16T00:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.616+0000 7fcb9cff9640 1 -- 172.21.15.31:0/813753918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcbb003b050 con 0x55bb5266b4a0 2024-09-16T00:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.616+0000 7fcbbdd58740 10 client.15555 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.616+0000 7fcbbdd58740 1 -- 172.21.15.31:0/813753918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bb52684c60 con 0x55bb5266b4a0 2024-09-16T00:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.623+0000 7fcb9cff9640 1 -- 172.21.15.31:0/813753918 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb00047a0 con 0x55bb5266b4a0 2024-09-16T00:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.623+0000 7fcb9cff9640 1 -- 172.21.15.31:0/813753918 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fcbb0003ce0 con 0x55bb5266b4a0 2024-09-16T00:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.624+0000 7fcbbdd58740 10 client.15555 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.624+0000 7fcbbdd58740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.624+0000 7fcbbdd58740 20 client.15555 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.624+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bb52686010 0x55bb52688400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.624+0000 7fcbbdd58740 4 client.15555 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.624+0000 7fcbbdd58740 1 -- 172.21.15.31:0/813753918 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb525de850 con 0x55bb52686010 2024-09-16T00:43:18.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.627+0000 7fcbb8f1a640 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bb52686010 0x55bb52688400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.635+0000 7fcbb8f1a640 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bb52686010 0x55bb52688400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fadf3ee4c1bb30aa server_cookie=704c43f27cd27708 in_seq=0 out_seq=0 2024-09-16T00:43:18.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.635+0000 7fcb9cff9640 10 client.15555 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.643+0000 7fcb9cff9640 1 -- 172.21.15.31:0/813753918 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb525de850 con 0x55bb52686010 2024-09-16T00:43:18.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.643+0000 7fcb9cff9640 10 client.15555 handle_command_reply: tid=0 2024-09-16T00:43:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.646+0000 7fcbbdd58740 1 client.15555 shutdown 2024-09-16T00:43:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.651+0000 7fcbbdd58740 1 -- 172.21.15.31:0/813753918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 msgr2=0x55bb52675f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.651+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb52675f90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0000c00 tx=0x7fcbb0009f60 comp rx=0 tx=0).stop 2024-09-16T00:43:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.655+0000 7fcbbdd58740 1 -- 172.21.15.31:0/813753918 shutdown_connections 2024-09-16T00:43:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.655+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb52676680 0x55bb52673780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.655+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bb52686010 0x55bb52688400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.655+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266bdc0 0x55bb52673240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.655+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/813753918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb52675f90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.655+0000 7fcbbdd58740 1 -- 172.21.15.31:0/813753918 >> 172.21.15.31:0/813753918 conn(0x55bb52667a30 msgr2=0x55bb5266d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.655+0000 7fcb9cff9640 0 client.15555 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:18.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.659+0000 7fcbbdd58740 1 -- 172.21.15.31:0/813753918 shutdown_connections 2024-09-16T00:43:18.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.663+0000 7fcbbdd58740 1 -- 172.21.15.31:0/813753918 wait complete. 2024-09-16T00:43:18.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.663+0000 7fcbbdd58740 20 client.15555 trim_cache size 0 max 16384 2024-09-16T00:43:18.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.708+0000 7fcbbdd58740 1 Processor -- start 2024-09-16T00:43:18.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.709+0000 7fcbbdd58740 1 -- start start 2024-09-16T00:43:18.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.709+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb52667a30 0x55bb52682540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.709+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb5266e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.710+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 0x55bb5268dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.710+0000 7fcbbdd58740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9800dca0 con 0x55bb5266b4a0 2024-09-16T00:43:18.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.710+0000 7fcbbdd58740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98010520 con 0x55bb5266bdc0 2024-09-16T00:43:18.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.710+0000 7fcbbdd58740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9800e310 con 0x55bb52667a30 2024-09-16T00:43:18.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.710+0000 7fcbb8f1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb5266e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.711+0000 7fcbb8f1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb5266e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39304/0 (socket says 172.21.15.31:39304) 2024-09-16T00:43:18.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.711+0000 7fcbb8f1a640 1 -- 172.21.15.31:0/233266966 learned_addr learned my addr 172.21.15.31:0/233266966 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:18.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.711+0000 7fcbb971b640 1 --2- 172.21.15.31:0/233266966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb52667a30 0x55bb52682540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.711+0000 7fcbb971b640 1 -- 172.21.15.31:0/233266966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 msgr2=0x55bb5268dc60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:43:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.711+0000 7fcbb971b640 1 --2- 172.21.15.31:0/233266966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 0x55bb5268dc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.711+0000 7fcbb971b640 1 -- 172.21.15.31:0/233266966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 msgr2=0x55bb5266e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.711+0000 7fcbb971b640 1 --2- 172.21.15.31:0/233266966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb5266e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.711+0000 7fcbb971b640 1 -- 172.21.15.31:0/233266966 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb52679410 con 0x55bb52667a30 2024-09-16T00:43:18.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.713+0000 7fcbb971b640 1 --2- 172.21.15.31:0/233266966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb52667a30 0x55bb52682540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0004ec0 tx=0x7fcbb000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d96aeb0f0f340ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:18.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.713+0000 7fcb9effd640 1 -- 172.21.15.31:0/233266966 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb001d040 con 0x55bb52667a30 2024-09-16T00:43:18.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.714+0000 7fcb9effd640 1 -- 172.21.15.31:0/233266966 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb0005590 con 0x55bb52667a30 2024-09-16T00:43:18.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.714+0000 7fcb9effd640 1 -- 172.21.15.31:0/233266966 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0023710 con 0x55bb52667a30 2024-09-16T00:43:18.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.716+0000 7fcbbdd58740 1 -- 172.21.15.31:0/233266966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb52667a30 msgr2=0x55bb52682540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:18.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.716+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/233266966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb52667a30 0x55bb52682540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0004ec0 tx=0x7fcbb000e720 comp rx=0 tx=0).stop 2024-09-16T00:43:18.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.718+0000 7fcbbdd58740 1 -- 172.21.15.31:0/233266966 shutdown_connections 2024-09-16T00:43:18.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.718+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/233266966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266bdc0 0x55bb5268dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.718+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/233266966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb5266b4a0 0x55bb5266e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.718+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/233266966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb52667a30 0x55bb52682540 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0004ec0 tx=0x7fcbb000e720 comp rx=0 tx=0).stop 2024-09-16T00:43:18.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.718+0000 7fcbbdd58740 1 -- 172.21.15.31:0/233266966 >> 172.21.15.31:0/233266966 conn(0x55bb526738e0 msgr2=0x55bb52666b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.723+0000 7fcbbdd58740 1 -- 172.21.15.31:0/233266966 shutdown_connections 2024-09-16T00:43:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.724+0000 7fcbbdd58740 1 -- 172.21.15.31:0/233266966 wait complete. 2024-09-16T00:43:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.725+0000 7fcbbdd58740 1 -- start start 2024-09-16T00:43:18.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.725+0000 7fcbbdd58740 1 Processor -- start 2024-09-16T00:43:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.726+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266b4a0 0x55bb5267e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.731+0000 7fcbbdd58740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266bdc0 0x55bb5267ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.731+0000 7fcbb971b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266b4a0 0x55bb5267e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.731+0000 7fcbb971b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266b4a0 0x55bb5267e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38388/0 (socket says 172.21.15.31:38388) 2024-09-16T00:43:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.732+0000 7fcbb971b640 1 -- 172.21.15.31:0/480738388 learned_addr learned my addr 172.21.15.31:0/480738388 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.732+0000 7fcbb8f1a640 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266bdc0 0x55bb5267ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.732+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 0x55bb5267b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.732+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9800e9e0 con 0x55bb52676680 2024-09-16T00:43:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.732+0000 7fcbb9f1c640 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 0x55bb5267b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.733+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9800fe50 con 0x55bb5266b4a0 2024-09-16T00:43:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.733+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9800f0b0 con 0x55bb5266bdc0 2024-09-16T00:43:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.733+0000 7fcbbdd58740 10 client.0 _ll_register_callbacks cb 0x7fcbb0009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:18.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.737+0000 7fcbb9f1c640 1 -- 172.21.15.31:0/480738388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266bdc0 msgr2=0x55bb5267ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:18.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.737+0000 7fcbb9f1c640 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266bdc0 0x55bb5267ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.737+0000 7fcbb9f1c640 1 -- 172.21.15.31:0/480738388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266b4a0 msgr2=0x55bb5267e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.737+0000 7fcbb9f1c640 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266b4a0 0x55bb5267e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.737+0000 7fcbb9f1c640 1 -- 172.21.15.31:0/480738388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5266b2d0 con 0x55bb52676680 2024-09-16T00:43:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.738+0000 7fcbb9f1c640 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 0x55bb5267b680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55bb52674740 tx=0x7fcbac037360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37a6b0366293356 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.738+0000 7fcb9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.738+0000 7fcb9cff9640 1 -- 172.21.15.31:0/480738388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac048060 con 0x55bb52676680 2024-09-16T00:43:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.738+0000 7fcb9cff9640 1 -- 172.21.15.31:0/480738388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fcbac030a70 con 0x55bb52676680 2024-09-16T00:43:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.738+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb0005590 con 0x55bb52676680 2024-09-16T00:43:18.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.739+0000 7fcb9cff9640 1 -- 172.21.15.31:0/480738388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcbac04f050 con 0x55bb52676680 2024-09-16T00:43:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.742+0000 7fcbbdd58740 10 client.15561 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.742+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bb52688a30 con 0x55bb52676680 2024-09-16T00:43:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.743+0000 7fcb9cff9640 1 -- 172.21.15.31:0/480738388 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbac003e30 con 0x55bb52676680 2024-09-16T00:43:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.743+0000 7fcb9cff9640 1 -- 172.21.15.31:0/480738388 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fcbac0377c0 con 0x55bb52676680 2024-09-16T00:43:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.743+0000 7fcbbdd58740 10 client.15561 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.743+0000 7fcbbdd58740 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:18.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.743+0000 7fcbbdd58740 20 client.15561 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.744+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bb526f1620 0x55bb526f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.744+0000 7fcbbdd58740 4 client.15561 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:43:18.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.744+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fcbb0005590 con 0x55bb526f1620 2024-09-16T00:43:18.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.748+0000 7fcbb971b640 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bb526f1620 0x55bb526f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:18.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.753+0000 7fcbb971b640 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bb526f1620 0x55bb526f3a40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1126e0b7da3b706f server_cookie=f74b8e4ba5abd5c5 in_seq=0 out_seq=0 2024-09-16T00:43:18.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.754+0000 7fcb9cff9640 10 client.15561 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:18.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.758+0000 7fcb9cff9640 1 -- 172.21.15.31:0/480738388 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fcbb0005590 con 0x55bb526f1620 2024-09-16T00:43:18.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.758+0000 7fcb9cff9640 10 client.15561 handle_command_reply: tid=0 2024-09-16T00:43:18.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.758+0000 7fcbbdd58740 1 client.15561 shutdown 2024-09-16T00:43:18.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.765+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 msgr2=0x55bb5267b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:18.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.765+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 0x55bb5267b680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55bb52674740 tx=0x7fcbac037360 comp rx=0 tx=0).stop 2024-09-16T00:43:18.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.765+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 shutdown_connections 2024-09-16T00:43:18.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.765+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb52676680 0x55bb5267b680 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.765+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bb526f1620 0x55bb526f3a40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.765+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb5266bdc0 0x55bb5267ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.765+0000 7fcbbdd58740 1 --2- 172.21.15.31:0/480738388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb5266b4a0 0x55bb5267e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.765+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 >> 172.21.15.31:0/480738388 conn(0x55bb526738e0 msgr2=0x55bb52666370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:18.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.765+0000 7fcb9cff9640 0 client.15561 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:18.772 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:43:18.772 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:43:18.772 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:43:18.772 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:43:18.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.771+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 shutdown_connections 2024-09-16T00:43:18.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.771+0000 7fcbbdd58740 1 -- 172.21.15.31:0/480738388 wait complete. 2024-09-16T00:43:18.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.771+0000 7fcbbdd58740 20 client.15561 trim_cache size 0 max 16384 2024-09-16T00:43:18.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.776+0000 7fcbbb97f640 1 -- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcba00a3460 msgr2=0x7fcba00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:18.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.776+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcba00a3460 0x7fcba00a5920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009ba0 tx=0x7fcbb000ef50 comp rx=0 tx=0).stop 2024-09-16T00:43:18.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.776+0000 7fcbbb97f640 1 -- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbb4109450 msgr2=0x7fcbb40761d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:18.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.776+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbb4109450 0x7fcbb40761d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006fd0 tx=0x7fcba400b8e0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.779+0000 7fcbbb97f640 1 -- 172.21.15.31:0/609548934 shutdown_connections 2024-09-16T00:43:18.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.779+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb4109e00 0x7fcbb4076710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.779+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbb4109450 0x7fcbb40761d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.779+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcba00a3460 0x7fcba00a5920 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.779+0000 7fcbbb97f640 1 --2- 172.21.15.31:0/609548934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbb4074990 0x7fcbb407b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:18.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.779+0000 7fcbbb97f640 1 -- 172.21.15.31:0/609548934 >> 172.21.15.31:0/609548934 conn(0x7fcbb406e140 msgr2=0x7fcbb4072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:18.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.781+0000 7fcbbb97f640 1 -- 172.21.15.31:0/609548934 shutdown_connections 2024-09-16T00:43:18.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:18.782+0000 7fcbbb97f640 1 -- 172.21.15.31:0/609548934 wait complete. 2024-09-16T00:43:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:19.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:19 smithi031 ceph-mon[30956]: pgmap v421: 129 pgs: 129 active+clean; 175 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 453 op/s 2024-09-16T00:43:19.682 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:43:19.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:43:19.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:19 smithi081 ceph-mon[35539]: pgmap v421: 129 pgs: 129 active+clean; 175 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 453 op/s 2024-09-16T00:43:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:19 smithi102 ceph-mon[35364]: pgmap v421: 129 pgs: 129 active+clean; 175 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 453 op/s 2024-09-16T00:43:20.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.184+0000 7f93ce48a640 1 -- 172.21.15.31:0/2718767862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102570 msgr2=0x7f93c81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.184+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2718767862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102570 0x7f93c81029b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009900 tx=0x7f93bc02a480 comp rx=0 tx=0).stop 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.184+0000 7f93ce48a640 1 -- 172.21.15.31:0/2718767862 shutdown_connections 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.184+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2718767862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c8102ef0 0x7f93c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.184+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2718767862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102570 0x7f93c81029b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.184+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2718767862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c8100e20 0x7f93c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.184+0000 7f93ce48a640 1 -- 172.21.15.31:0/2718767862 >> 172.21.15.31:0/2718767862 conn(0x7f93c80fc7d0 msgr2=0x7f93c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.184+0000 7f93ce48a640 1 -- 172.21.15.31:0/2718767862 shutdown_connections 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.184+0000 7f93ce48a640 1 -- 172.21.15.31:0/2718767862 wait complete. 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93ce48a640 1 Processor -- start 2024-09-16T00:43:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93ce48a640 1 -- start start 2024-09-16T00:43:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93ce48a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c8100e20 0x7f93c80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93ce48a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c8102570 0x7f93c80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93ce48a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102ef0 0x7f93c80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93ce48a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8110f80 con 0x7f93c8102ef0 2024-09-16T00:43:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93ce48a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8110e00 con 0x7f93c8102570 2024-09-16T00:43:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93ce48a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8111100 con 0x7f93c8100e20 2024-09-16T00:43:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93cca27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102ef0 0x7f93c80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c8102570 0x7f93c80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93cca27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102ef0 0x7f93c80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39326/0 (socket says 172.21.15.31:39326) 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93cca27640 1 -- 172.21.15.31:0/2208371458 learned_addr learned my addr 172.21.15.31:0/2208371458 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.185+0000 7f93c7fff640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c8100e20 0x7f93c80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93c77fe640 1 -- 172.21.15.31:0/2208371458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c8100e20 msgr2=0x7f93c80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93c77fe640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c8100e20 0x7f93c80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93c77fe640 1 -- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102ef0 msgr2=0x7f93c80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93c77fe640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102ef0 0x7f93c80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93c77fe640 1 -- 172.21.15.31:0/2208371458 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c81a6970 con 0x7f93c8102570 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93cca27640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102ef0 0x7f93c80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93c77fe640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c8102570 0x7f93c80fd1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93bc005940 tx=0x7f93bc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93c57fa640 1 -- 172.21.15.31:0/2208371458 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc03f070 con 0x7f93c8102570 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93ce48a640 1 -- 172.21.15.31:0/2208371458 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c81a6c00 con 0x7f93c8102570 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93c57fa640 1 -- 172.21.15.31:0/2208371458 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93bc031c30 con 0x7f93c8102570 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93c57fa640 1 -- 172.21.15.31:0/2208371458 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc035030 con 0x7f93c8102570 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.186+0000 7f93ce48a640 1 -- 172.21.15.31:0/2208371458 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c81a7130 con 0x7f93c8102570 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93d0863740 1 Processor -- start 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93d0863740 1 -- start start 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93c57fa640 1 -- 172.21.15.31:0/2208371458 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f93bc042070 con 0x7f93c8102570 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93d0863740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db912fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93d0863740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db9130170 0x555db913a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93d0863740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db913aa30 0x555db913ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93d0863740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555db9094160 con 0x555db912f850 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93d0863740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555db903b720 con 0x555db9130170 2024-09-16T00:43:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93d0863740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555db9090120 con 0x555db913aa30 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93cca27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db912fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93cca27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db912fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39340/0 (socket says 172.21.15.31:39340) 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93cca27640 1 -- 172.21.15.31:0/335410152 learned_addr learned my addr 172.21.15.31:0/335410152 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93c7fff640 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db9130170 0x555db913a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93c77fe640 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db913aa30 0x555db913ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93cca27640 1 -- 172.21.15.31:0/335410152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db913aa30 msgr2=0x555db913ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93cca27640 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db913aa30 0x555db913ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93cca27640 1 -- 172.21.15.31:0/335410152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db9130170 msgr2=0x555db913a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.189+0000 7f93cca27640 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db9130170 0x555db913a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.190+0000 7f93cca27640 1 -- 172.21.15.31:0/335410152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db8e05ba0 con 0x555db912f850 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.190+0000 7f93c7fff640 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db9130170 0x555db913a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.190+0000 7f93cca27640 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db912fc30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f93c0011850 tx=0x7f93c0011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aabe652ac8d6a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.190+0000 7f93c57fa640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93b00a3240 0x7f93b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.190+0000 7f93c7fff640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93b00a3240 0x7f93b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.190+0000 7f93c57fa640 1 -- 172.21.15.31:0/2208371458 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f93bc0f02c0 con 0x7f93c8102570 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.190+0000 7f93ae7fc640 1 -- 172.21.15.31:0/335410152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0011330 con 0x555db912f850 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.190+0000 7f93ae7fc640 1 -- 172.21.15.31:0/335410152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c000fe80 con 0x555db912f850 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.190+0000 7f93ae7fc640 1 -- 172.21.15.31:0/335410152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c00193e0 con 0x555db912f850 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93d0863740 1 -- 172.21.15.31:0/335410152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 msgr2=0x555db912fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93c7fff640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93b00a3240 0x7f93b00a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x555db912cc40 tx=0x7f93b800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93d0863740 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db912fc30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f93c0011850 tx=0x7f93c0011d10 comp rx=0 tx=0).stop 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93d0863740 1 -- 172.21.15.31:0/335410152 shutdown_connections 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93d0863740 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db913aa30 0x555db913ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93d0863740 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db9130170 0x555db913a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93d0863740 1 --2- 172.21.15.31:0/335410152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db912fc30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93d0863740 1 -- 172.21.15.31:0/335410152 >> 172.21.15.31:0/335410152 conn(0x555db912bde0 msgr2=0x555db912c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:20.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93d0863740 1 -- 172.21.15.31:0/335410152 shutdown_connections 2024-09-16T00:43:20.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.191+0000 7f93d0863740 1 -- 172.21.15.31:0/335410152 wait complete. 2024-09-16T00:43:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.192+0000 7f93d0863740 1 -- start start 2024-09-16T00:43:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.192+0000 7f93d0863740 1 Processor -- start 2024-09-16T00:43:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.192+0000 7f93d0863740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db912f850 0x555db9137840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.192+0000 7f93d0863740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db9130170 0x555db9137d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.192+0000 7f93d0863740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 0x555db91382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.192+0000 7f93d0863740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555db913d600 con 0x555db9130170 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.192+0000 7f93d0863740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555db901c900 con 0x555db913aa30 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.192+0000 7f93d0863740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555db913d780 con 0x555db912f850 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.192+0000 7f93d0863740 10 client.0 _ll_register_callbacks cb 0x555db901a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db9130170 0x555db9137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93cca27640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 0x555db91382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93cca27640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 0x555db91382c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38434/0 (socket says 172.21.15.31:38434) 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93cca27640 1 -- 172.21.15.31:0/3560069454 learned_addr learned my addr 172.21.15.31:0/3560069454 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db9130170 0x555db9137d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39354/0 (socket says 172.21.15.31:39354) 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93c7fff640 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db912f850 0x555db9137840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93cca27640 1 -- 172.21.15.31:0/3560069454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db912f850 msgr2=0x555db9137840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93cca27640 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db912f850 0x555db9137840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93cca27640 1 -- 172.21.15.31:0/3560069454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db9130170 msgr2=0x555db9137d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93cca27640 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db9130170 0x555db9137d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93cca27640 1 -- 172.21.15.31:0/3560069454 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db905cc50 con 0x555db913aa30 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93c77fe640 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db9130170 0x555db9137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93cca27640 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 0x555db91382c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009a70 tx=0x7f93c0011450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69742eb4ef7dd3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93a77fe640 1 -- 172.21.15.31:0/3560069454 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c001f040 con 0x555db913aa30 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93a77fe640 1 -- 172.21.15.31:0/3560069454 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f93c0004a40 con 0x555db913aa30 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.193+0000 7f93d0863740 1 -- 172.21.15.31:0/3560069454 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555db90a1890 con 0x555db913aa30 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93a77fe640 1 -- 172.21.15.31:0/3560069454 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f93c0032050 con 0x555db913aa30 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93d0863740 10 client.25262 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93d0863740 1 -- 172.21.15.31:0/3560069454 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555db9149200 con 0x555db913aa30 2024-09-16T00:43:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93a77fe640 1 -- 172.21.15.31:0/3560069454 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555db90a1890 con 0x555db913aa30 2024-09-16T00:43:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93a77fe640 1 -- 172.21.15.31:0/3560069454 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f93c0018070 con 0x555db913aa30 2024-09-16T00:43:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93d0863740 10 client.25262 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93d0863740 10 client.25262 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:43:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93d0863740 20 client.25262 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93d0863740 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555db914a660 0x555db914ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93d0863740 4 client.25262 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93d0863740 1 -- 172.21.15.31:0/3560069454 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555db909bbe0 con 0x555db914a660 2024-09-16T00:43:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.194+0000 7f93c7fff640 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555db914a660 0x555db914ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.195+0000 7f93a77fe640 10 client.25262 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.195+0000 7f93c7fff640 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555db914a660 0x555db914ca50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17711ce5a8ae67b server_cookie=a9c86bc52317b14a in_seq=0 out_seq=0 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93a77fe640 1 -- 172.21.15.31:0/3560069454 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555db909bbe0 con 0x555db914a660 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93a77fe640 10 client.25262 handle_command_reply: tid=0 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 client.25262 shutdown 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 -- 172.21.15.31:0/3560069454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 msgr2=0x555db91382c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 0x555db91382c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009a70 tx=0x7f93c0011450 comp rx=0 tx=0).stop 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 -- 172.21.15.31:0/3560069454 shutdown_connections 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 0x555db91382c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db9130170 0x555db9137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555db914a660 0x555db914ca50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 --2- 172.21.15.31:0/3560069454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db912f850 0x555db9137840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 -- 172.21.15.31:0/3560069454 >> 172.21.15.31:0/3560069454 conn(0x555db912bde0 msgr2=0x555db9131b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93a77fe640 0 client.25262 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 -- 172.21.15.31:0/3560069454 shutdown_connections 2024-09-16T00:43:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 1 -- 172.21.15.31:0/3560069454 wait complete. 2024-09-16T00:43:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.198+0000 7f93d0863740 20 client.25262 trim_cache size 0 max 16384 2024-09-16T00:43:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.217+0000 7f93d0863740 1 Processor -- start 2024-09-16T00:43:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.217+0000 7f93d0863740 1 -- start start 2024-09-16T00:43:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.217+0000 7f93d0863740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db912f850 0x555db9126f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.217+0000 7f93d0863740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 0x555db9152170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.217+0000 7f93d0863740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db913aa30 0x555db91526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.217+0000 7f93d0863740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93c81aeed0 con 0x555db913aa30 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.217+0000 7f93d0863740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93c81b1750 con 0x555db912f850 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.217+0000 7f93d0863740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93c81af540 con 0x555db9130170 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db912f850 0x555db9126f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db912f850 0x555db9126f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38446/0 (socket says 172.21.15.31:38446) 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c7fff640 1 -- 172.21.15.31:0/869146635 learned_addr learned my addr 172.21.15.31:0/869146635 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c77fe640 1 --2- 172.21.15.31:0/869146635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 0x555db9152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93cca27640 1 --2- 172.21.15.31:0/869146635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db913aa30 0x555db91526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c7fff640 1 -- 172.21.15.31:0/869146635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 msgr2=0x555db9152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c7fff640 1 --2- 172.21.15.31:0/869146635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 0x555db9152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c7fff640 1 -- 172.21.15.31:0/869146635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db913aa30 msgr2=0x555db91526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c7fff640 1 --2- 172.21.15.31:0/869146635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db913aa30 0x555db91526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c7fff640 1 -- 172.21.15.31:0/869146635 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db912f680 con 0x555db912f850 2024-09-16T00:43:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93c7fff640 1 --2- 172.21.15.31:0/869146635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db912f850 0x555db9126f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f93b8003ee0 tx=0x7f93b8019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad9486f32e2cb95 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:20.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.218+0000 7f93adffb640 1 -- 172.21.15.31:0/869146635 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b801eaa0 con 0x555db912f850 2024-09-16T00:43:20.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.219+0000 7f93adffb640 1 -- 172.21.15.31:0/869146635 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b801ec40 con 0x555db912f850 2024-09-16T00:43:20.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.219+0000 7f93adffb640 1 -- 172.21.15.31:0/869146635 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b801d830 con 0x555db912f850 2024-09-16T00:43:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.219+0000 7f93d0863740 1 -- 172.21.15.31:0/869146635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db912f850 msgr2=0x555db9126f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.219+0000 7f93d0863740 1 --2- 172.21.15.31:0/869146635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db912f850 0x555db9126f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f93b8003ee0 tx=0x7f93b8019000 comp rx=0 tx=0).stop 2024-09-16T00:43:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.219+0000 7f93d0863740 1 -- 172.21.15.31:0/869146635 shutdown_connections 2024-09-16T00:43:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.219+0000 7f93d0863740 1 --2- 172.21.15.31:0/869146635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db913aa30 0x555db91526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.220+0000 7f93d0863740 1 --2- 172.21.15.31:0/869146635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 0x555db9152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.220+0000 7f93d0863740 1 --2- 172.21.15.31:0/869146635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db912f850 0x555db9126f60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.220+0000 7f93d0863740 1 -- 172.21.15.31:0/869146635 >> 172.21.15.31:0/869146635 conn(0x555db912bde0 msgr2=0x555db9131ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.220+0000 7f93d0863740 1 -- 172.21.15.31:0/869146635 shutdown_connections 2024-09-16T00:43:20.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.220+0000 7f93d0863740 1 -- 172.21.15.31:0/869146635 wait complete. 2024-09-16T00:43:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.220+0000 7f93d0863740 1 -- start start 2024-09-16T00:43:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.220+0000 7f93d0863740 1 Processor -- start 2024-09-16T00:43:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93d0863740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db9142e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93d0863740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 0x555db913e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93d0863740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 0x555db913e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93d0863740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93c81afc10 con 0x555db912f850 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93d0863740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93c81b1080 con 0x555db913aa30 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93d0863740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93c81b02e0 con 0x555db9130170 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db9142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93d0863740 10 client.0 _ll_register_callbacks cb 0x7f93c0011570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db9142e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39372/0 (socket says 172.21.15.31:39372) 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93c7fff640 1 -- 172.21.15.31:0/4047290895 learned_addr learned my addr 172.21.15.31:0/4047290895 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93c77fe640 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 0x555db913e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93c7fff640 1 -- 172.21.15.31:0/4047290895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 msgr2=0x555db913e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93c7fff640 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 0x555db913e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.221+0000 7f93c7fff640 1 -- 172.21.15.31:0/4047290895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 msgr2=0x555db913e6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:43:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.222+0000 7f93c7fff640 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 0x555db913e6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.222+0000 7f93c7fff640 1 -- 172.21.15.31:0/4047290895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db912ce90 con 0x555db912f850 2024-09-16T00:43:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.222+0000 7f93c7fff640 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db9142e80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f93b801e7f0 tx=0x7f93b80042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf117e9d2edc0335 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.222+0000 7f93a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.222+0000 7f93a77fe640 1 -- 172.21.15.31:0/4047290895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b802a420 con 0x555db912f850 2024-09-16T00:43:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93a77fe640 1 -- 172.21.15.31:0/4047290895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f93b802a5c0 con 0x555db912f850 2024-09-16T00:43:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93d0863740 1 -- 172.21.15.31:0/4047290895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b801ec40 con 0x555db912f850 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93a77fe640 1 -- 172.21.15.31:0/4047290895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f93b802a7c0 con 0x555db912f850 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93d0863740 10 client.15579 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93d0863740 1 -- 172.21.15.31:0/4047290895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555db91bd490 con 0x555db912f850 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93a77fe640 1 -- 172.21.15.31:0/4047290895 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b801ec40 con 0x555db912f850 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93a77fe640 1 -- 172.21.15.31:0/4047290895 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f93b803c020 con 0x555db912f850 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93d0863740 10 client.15579 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93d0863740 10 client.15579 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93d0863740 20 client.15579 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93d0863740 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555db91bdb60 0x555db91bff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93d0863740 4 client.15579 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-16T00:43:20.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.223+0000 7f93d0863740 1 -- 172.21.15.31:0/4047290895 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x555db9017f40 con 0x555db91bdb60 2024-09-16T00:43:20.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.224+0000 7f93c77fe640 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555db91bdb60 0x555db91bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.224+0000 7f93a77fe640 10 client.15579 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:20.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.224+0000 7f93c77fe640 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555db91bdb60 0x555db91bff80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45124827d2365f5d server_cookie=5cc0d674fd7d428f in_seq=0 out_seq=0 2024-09-16T00:43:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.819+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2555492621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 msgr2=0x7f0f541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.819+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2555492621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 0x7f0f541029b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009900 tx=0x7f0f3c02a480 comp rx=0 tx=0).stop 2024-09-16T00:43:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.819+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2555492621 shutdown_connections 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.819+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2555492621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54102ef0 0x7f0f5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.819+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2555492621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 0x7f0f541029b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.819+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2555492621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f54100e20 0x7f0f54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.819+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2555492621 >> 172.21.15.31:0/2555492621 conn(0x7f0f540fc7d0 msgr2=0x7f0f540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.819+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2555492621 shutdown_connections 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.819+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2555492621 wait complete. 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f59cb5640 1 Processor -- start 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f59cb5640 1 -- start start 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f59cb5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54100e20 0x7f0f54199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f59cb5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 0x7f0f54199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f59cb5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f54102ef0 0x7f0f541a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f59cb5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54110f80 con 0x7f0f54102570 2024-09-16T00:43:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f59cb5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54110e00 con 0x7f0f54102ef0 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f59cb5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54111100 con 0x7f0f54100e20 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f52ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 0x7f0f54199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f53fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f54102ef0 0x7f0f541a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f52ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 0x7f0f54199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39388/0 (socket says 172.21.15.31:39388) 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f52ffd640 1 -- 172.21.15.31:0/2031683584 learned_addr learned my addr 172.21.15.31:0/2031683584 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.820+0000 7f0f537fe640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54100e20 0x7f0f54199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f53fff640 1 -- 172.21.15.31:0/2031683584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54100e20 msgr2=0x7f0f54199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f53fff640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54100e20 0x7f0f54199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f53fff640 1 -- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 msgr2=0x7f0f54199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f53fff640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 0x7f0f54199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f53fff640 1 -- 172.21.15.31:0/2031683584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f541a4910 con 0x7f0f54102ef0 2024-09-16T00:43:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f537fe640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54100e20 0x7f0f54199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f52ffd640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 0x7f0f54199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f53fff640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f54102ef0 0x7f0f541a43d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00e970 tx=0x7f0f4c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f50ff9640 1 -- 172.21.15.31:0/2031683584 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00ccc0 con 0x7f0f54102ef0 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f541a4ae0 con 0x7f0f54102ef0 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f50ff9640 1 -- 172.21.15.31:0/2031683584 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c005020 con 0x7f0f54102ef0 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f50ff9640 1 -- 172.21.15.31:0/2031683584 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c004a00 con 0x7f0f54102ef0 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.821+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f541a5020 con 0x7f0f54102ef0 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.822+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f54101200 con 0x7f0f54102ef0 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.823+0000 7f0f50ff9640 1 -- 172.21.15.31:0/2031683584 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0f4c0051d0 con 0x7f0f54102ef0 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.824+0000 7f0f50ff9640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0f400a3240 0x7f0f400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.824+0000 7f0f537fe640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0f400a3240 0x7f0f400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:20.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.824+0000 7f0f50ff9640 1 -- 172.21.15.31:0/2031683584 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f0f4c0cfeb0 con 0x7f0f54102ef0 2024-09-16T00:43:20.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.825+0000 7f0f537fe640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0f400a3240 0x7f0f400a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009fd0 tx=0x7f0f44009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:20.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:20.827+0000 7f0f50ff9640 1 -- 172.21.15.31:0/2031683584 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f4c09c7c0 con 0x7f0f54102ef0 2024-09-16T00:43:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.041+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f54109090 con 0x7f0f54102ef0 2024-09-16T00:43:21.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.042+0000 7f0f50ff9640 1 -- 172.21.15.31:0/2031683584 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f0f4c09c0c0 con 0x7f0f54102ef0 2024-09-16T00:43:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:43:21.043 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:43:21.043 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:43:21.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0f400a3240 msgr2=0x7f0f400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:21.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0f400a3240 0x7f0f400a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009fd0 tx=0x7f0f44009760 comp rx=0 tx=0).stop 2024-09-16T00:43:21.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f54102ef0 msgr2=0x7f0f541a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:21.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f54102ef0 0x7f0f541a43d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00e970 tx=0x7f0f4c00ee30 comp rx=0 tx=0).stop 2024-09-16T00:43:21.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 shutdown_connections 2024-09-16T00:43:21.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f54102ef0 0x7f0f541a43d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:21.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f54102570 0x7f0f54199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:21.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f0f400a3240 0x7f0f400a5700 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:21.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 --2- 172.21.15.31:0/2031683584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54100e20 0x7f0f54199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:21.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 >> 172.21.15.31:0/2031683584 conn(0x7f0f540fc7d0 msgr2=0x7f0f541068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:21.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 shutdown_connections 2024-09-16T00:43:21.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:21.045+0000 7f0f59cb5640 1 -- 172.21.15.31:0/2031683584 wait complete. 2024-09-16T00:43:21.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:21 smithi031 ceph-mon[30956]: pgmap v422: 129 pgs: 129 active+clean; 185 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-09-16T00:43:21.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:21 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2031683584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:21.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:43:21.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:21 smithi081 ceph-mon[35539]: pgmap v422: 129 pgs: 129 active+clean; 185 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-09-16T00:43:21.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:21 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2031683584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:21.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:21 smithi102 ceph-mon[35364]: pgmap v422: 129 pgs: 129 active+clean; 185 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-09-16T00:43:21.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:21 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2031683584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.530+0000 7f559fd05640 1 -- 172.21.15.31:0/2761513618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5598074eb0 msgr2=0x7f5598112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.530+0000 7f559fd05640 1 --2- 172.21.15.31:0/2761513618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5598074eb0 0x7f5598112640 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f558c009900 tx=0x7f558c02a480 comp rx=0 tx=0).stop 2024-09-16T00:43:22.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.530+0000 7f559fd05640 1 -- 172.21.15.31:0/2761513618 shutdown_connections 2024-09-16T00:43:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.530+0000 7f559fd05640 1 --2- 172.21.15.31:0/2761513618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5598112b80 0x7f5598114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.530+0000 7f559fd05640 1 --2- 172.21.15.31:0/2761513618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5598074eb0 0x7f5598112640 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.531+0000 7f559fd05640 1 --2- 172.21.15.31:0/2761513618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5598074590 0x7f5598074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.531+0000 7f559fd05640 1 -- 172.21.15.31:0/2761513618 >> 172.21.15.31:0/2761513618 conn(0x7f55980fd8b0 msgr2=0x7f55980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.531+0000 7f559fd05640 1 -- 172.21.15.31:0/2761513618 shutdown_connections 2024-09-16T00:43:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.531+0000 7f559fd05640 1 -- 172.21.15.31:0/2761513618 wait complete. 2024-09-16T00:43:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559fd05640 1 Processor -- start 2024-09-16T00:43:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559fd05640 1 -- start start 2024-09-16T00:43:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559fd05640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5598074590 0x7f55981121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559fd05640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5598074eb0 0x7f559810b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559fd05640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5598112b80 0x7f559810b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559fd05640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5598118530 con 0x7f5598112b80 2024-09-16T00:43:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559fd05640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55981183b0 con 0x7f5598074eb0 2024-09-16T00:43:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559e2a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5598112b80 0x7f559810b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559daa1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5598074590 0x7f55981121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559d2a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5598074eb0 0x7f559810b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559daa1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5598074590 0x7f55981121b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:60472/0 (socket says 172.21.15.31:60472) 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559d2a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5598074eb0 0x7f559810b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38472/0 (socket says 172.21.15.31:38472) 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559daa1640 1 -- 172.21.15.31:0/3738301049 learned_addr learned my addr 172.21.15.31:0/3738301049 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.532+0000 7f559fd05640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55981186b0 con 0x7f5598074590 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.533+0000 7f559d2a0640 1 -- 172.21.15.31:0/3738301049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5598074590 msgr2=0x7f55981121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.533+0000 7f559d2a0640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5598074590 0x7f55981121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.533+0000 7f559d2a0640 1 -- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5598112b80 msgr2=0x7f559810b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.533+0000 7f559d2a0640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5598112b80 0x7f559810b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.533+0000 7f559d2a0640 1 -- 172.21.15.31:0/3738301049 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559810bd90 con 0x7f5598074eb0 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.533+0000 7f559e2a2640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5598112b80 0x7f559810b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.533+0000 7f559d2a0640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5598074eb0 0x7f559810b240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f558c005940 tx=0x7f558c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.533+0000 7f558affd640 1 -- 172.21.15.31:0/3738301049 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c03f030 con 0x7f5598074eb0 2024-09-16T00:43:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.534+0000 7f558affd640 1 -- 172.21.15.31:0/3738301049 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c0044c0 con 0x7f5598074eb0 2024-09-16T00:43:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.534+0000 7f558affd640 1 -- 172.21.15.31:0/3738301049 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c035070 con 0x7f5598074eb0 2024-09-16T00:43:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.534+0000 7f559fd05640 1 -- 172.21.15.31:0/3738301049 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559810c020 con 0x7f5598074eb0 2024-09-16T00:43:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.534+0000 7f559fd05640 1 -- 172.21.15.31:0/3738301049 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559810ff70 con 0x7f5598074eb0 2024-09-16T00:43:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.537+0000 7f55a20de740 1 Processor -- start 2024-09-16T00:43:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f55a20de740 1 -- start start 2024-09-16T00:43:22.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f558affd640 1 -- 172.21.15.31:0/3738301049 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f558c042030 con 0x7f5598074eb0 2024-09-16T00:43:22.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f55a20de740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a40b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f55a20de740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40b5d0 0x56233a40ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f55a20de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a418760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f55a20de740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56233a3712f0 con 0x56233a40b5d0 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f55a20de740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56233a318cf0 con 0x56233a40cb40 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f55a20de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56233a36d2b0 con 0x56233a416370 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f559e2a2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a40b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f559e2a2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a40b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38482/0 (socket says 172.21.15.31:38482) 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f559e2a2640 1 -- 172.21.15.31:0/2078851288 learned_addr learned my addr 172.21.15.31:0/2078851288 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f559daa1640 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40b5d0 0x56233a40ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.538+0000 7f559d2a0640 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a418760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f559e2a2640 1 -- 172.21.15.31:0/2078851288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 msgr2=0x56233a418760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f559e2a2640 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a418760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f559e2a2640 1 -- 172.21.15.31:0/2078851288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40b5d0 msgr2=0x56233a40ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f559e2a2640 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40b5d0 0x56233a40ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f559e2a2640 1 -- 172.21.15.31:0/2078851288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233a2fd3a0 con 0x56233a40cb40 2024-09-16T00:43:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f559d2a0640 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a418760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f559e2a2640 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a40b090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5590011760 tx=0x7f5590011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb6c0352531c0c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f5583fff640 1 -- 172.21.15.31:0/2078851288 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590004d20 con 0x56233a40cb40 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f5583fff640 1 -- 172.21.15.31:0/2078851288 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f559000f9a0 con 0x56233a40cb40 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.539+0000 7f5583fff640 1 -- 172.21.15.31:0/2078851288 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590004590 con 0x56233a40cb40 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.540+0000 7f558affd640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55840a3410 0x7f55840a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.540+0000 7f559daa1640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55840a3410 0x7f55840a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.540+0000 7f55a20de740 1 -- 172.21.15.31:0/2078851288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 msgr2=0x56233a40b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.540+0000 7f55a20de740 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a40b090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5590011760 tx=0x7f5590011c20 comp rx=0 tx=0).stop 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.540+0000 7f55a20de740 1 -- 172.21.15.31:0/2078851288 shutdown_connections 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.540+0000 7f55a20de740 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a418760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.541+0000 7f55a20de740 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40b5d0 0x56233a40ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.541+0000 7f55a20de740 1 --2- 172.21.15.31:0/2078851288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a40b090 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.541+0000 7f55a20de740 1 -- 172.21.15.31:0/2078851288 >> 172.21.15.31:0/2078851288 conn(0x56233a3d4310 msgr2=0x56233a407c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.541+0000 7f558affd640 1 -- 172.21.15.31:0/3738301049 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f558c0eff80 con 0x7f5598074eb0 2024-09-16T00:43:22.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.541+0000 7f55a20de740 1 -- 172.21.15.31:0/2078851288 shutdown_connections 2024-09-16T00:43:22.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.541+0000 7f55a20de740 1 -- 172.21.15.31:0/2078851288 wait complete. 2024-09-16T00:43:22.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.541+0000 7f55a20de740 1 -- start start 2024-09-16T00:43:22.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.541+0000 7f55a20de740 1 Processor -- start 2024-09-16T00:43:22.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f559daa1640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55840a3410 0x7f55840a58d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56233a408ac0 tx=0x7f5594005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:22.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f55a20de740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40b5d0 0x56233a415d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f55a20de740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a413030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f55a20de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a413640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f55a20de740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56233a419070 con 0x56233a40cb40 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f55a20de740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56233a32c060 con 0x56233a40b5d0 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f55a20de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56233a4191f0 con 0x56233a416370 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f559d2a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a413030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f55a20de740 10 client.0 _ll_register_callbacks cb 0x56233a07f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f559e2a2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a413640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f559d2a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a413030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39428/0 (socket says 172.21.15.31:39428) 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f559d2a0640 1 -- 172.21.15.31:0/215152224 learned_addr learned my addr 172.21.15.31:0/215152224 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.542+0000 7f559daa1640 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40b5d0 0x56233a415d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.543+0000 7f559d2a0640 1 -- 172.21.15.31:0/215152224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 msgr2=0x56233a413640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.543+0000 7f559d2a0640 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a413640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.543+0000 7f559d2a0640 1 -- 172.21.15.31:0/215152224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40b5d0 msgr2=0x56233a415d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.543+0000 7f559d2a0640 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40b5d0 0x56233a415d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.543+0000 7f559d2a0640 1 -- 172.21.15.31:0/215152224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233a0e23c0 con 0x56233a40cb40 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.543+0000 7f559daa1640 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40b5d0 0x56233a415d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.543+0000 7f559e2a2640 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a413640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.543+0000 7f559d2a0640 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a413030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f558c0b8b60 tx=0x7f558c0b8430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bb33cdb39fcbe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.543+0000 7f55817fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.544+0000 7f55817fa640 1 -- 172.21.15.31:0/215152224 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c049070 con 0x56233a40cb40 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.544+0000 7f55817fa640 1 -- 172.21.15.31:0/215152224 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f558c03bb70 con 0x56233a40cb40 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.544+0000 7f55a20de740 1 -- 172.21.15.31:0/215152224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559000f9a0 con 0x56233a40cb40 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.544+0000 7f55817fa640 1 -- 172.21.15.31:0/215152224 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f558c0bbaf0 con 0x56233a40cb40 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.544+0000 7f55a20de740 10 client.15597 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.544+0000 7f55a20de740 1 -- 172.21.15.31:0/215152224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56233a424b70 con 0x56233a40cb40 2024-09-16T00:43:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.544+0000 7f55817fa640 1 -- 172.21.15.31:0/215152224 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f559000f9a0 con 0x56233a40cb40 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.545+0000 7f55817fa640 1 -- 172.21.15.31:0/215152224 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f558c0b8800 con 0x56233a40cb40 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.545+0000 7f55a20de740 10 client.15597 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.545+0000 7f55a20de740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.545+0000 7f55a20de740 20 client.15597 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.545+0000 7f55a20de740 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56233a4260c0 0x56233a428520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.545+0000 7f55a20de740 4 client.15597 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.545+0000 7f55a20de740 1 -- 172.21.15.31:0/215152224 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56233a37e860 con 0x56233a4260c0 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.545+0000 7f559daa1640 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56233a4260c0 0x56233a428520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.546+0000 7f559daa1640 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56233a4260c0 0x56233a428520 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b4b0db94bccc5b server_cookie=7581c913616be19 in_seq=0 out_seq=0 2024-09-16T00:43:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.546+0000 7f55817fa640 10 client.15597 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55817fa640 1 -- 172.21.15.31:0/215152224 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56233a37e860 con 0x56233a4260c0 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55817fa640 10 client.15597 handle_command_reply: tid=0 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 client.15597 shutdown 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 -- 172.21.15.31:0/215152224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 msgr2=0x56233a413030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a413030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f558c0b8b60 tx=0x7f558c0b8430 comp rx=0 tx=0).stop 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 -- 172.21.15.31:0/215152224 shutdown_connections 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56233a4260c0 0x56233a428520 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a416370 0x56233a413640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a413030 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 --2- 172.21.15.31:0/215152224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40b5d0 0x56233a415d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 -- 172.21.15.31:0/215152224 >> 172.21.15.31:0/215152224 conn(0x56233a3d4310 msgr2=0x56233a40d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55817fa640 0 client.15597 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 -- 172.21.15.31:0/215152224 shutdown_connections 2024-09-16T00:43:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.549+0000 7f55a20de740 1 -- 172.21.15.31:0/215152224 wait complete. 2024-09-16T00:43:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.550+0000 7f55a20de740 20 client.15597 trim_cache size 0 max 16384 2024-09-16T00:43:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.563+0000 7f55a20de740 1 Processor -- start 2024-09-16T00:43:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.563+0000 7f55a20de740 1 -- start start 2024-09-16T00:43:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.563+0000 7f55a20de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 0x56233a422340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.563+0000 7f55a20de740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a40e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.563+0000 7f55a20de740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a416370 0x56233a40e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.563+0000 7f55a20de740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55981bbbe0 con 0x56233a40cb40 2024-09-16T00:43:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.563+0000 7f55a20de740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55981be460 con 0x56233a416370 2024-09-16T00:43:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.563+0000 7f55a20de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55981bc250 con 0x56233a40b5d0 2024-09-16T00:43:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559d2a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a40e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559d2a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a40e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39436/0 (socket says 172.21.15.31:39436) 2024-09-16T00:43:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559d2a0640 1 -- 172.21.15.31:0/6154752 learned_addr learned my addr 172.21.15.31:0/6154752 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559daa1640 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 0x56233a422340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559e2a2640 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a416370 0x56233a40e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559d2a0640 1 -- 172.21.15.31:0/6154752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 msgr2=0x56233a422340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559d2a0640 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 0x56233a422340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559d2a0640 1 -- 172.21.15.31:0/6154752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a416370 msgr2=0x56233a40e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559d2a0640 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a416370 0x56233a40e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559d2a0640 1 -- 172.21.15.31:0/6154752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233a413fd0 con 0x56233a40cb40 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559daa1640 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 0x56233a422340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559e2a2640 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a416370 0x56233a40e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.564+0000 7f559d2a0640 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a40e120 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f558c009900 tx=0x7f558c0c0540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79784ae9ab1d2a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.565+0000 7f55837fe640 1 -- 172.21.15.31:0/6154752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c049070 con 0x56233a40cb40 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.565+0000 7f55837fe640 1 -- 172.21.15.31:0/6154752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c0d5170 con 0x56233a40cb40 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.565+0000 7f55837fe640 1 -- 172.21.15.31:0/6154752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c049070 con 0x56233a40cb40 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 -- 172.21.15.31:0/6154752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 msgr2=0x56233a40e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a40e120 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f558c009900 tx=0x7f558c0c0540 comp rx=0 tx=0).stop 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 -- 172.21.15.31:0/6154752 shutdown_connections 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a416370 0x56233a40e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a40cb40 0x56233a40e120 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 --2- 172.21.15.31:0/6154752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 0x56233a422340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 -- 172.21.15.31:0/6154752 >> 172.21.15.31:0/6154752 conn(0x56233a3d4310 msgr2=0x56233a412120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 -- 172.21.15.31:0/6154752 shutdown_connections 2024-09-16T00:43:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 -- 172.21.15.31:0/6154752 wait complete. 2024-09-16T00:43:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 -- start start 2024-09-16T00:43:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.566+0000 7f55a20de740 1 Processor -- start 2024-09-16T00:43:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f55a20de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 0x56233a41e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f55a20de740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a41ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f55a20de740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a416370 0x56233a40f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f55a20de740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55981bc920 con 0x56233a416370 2024-09-16T00:43:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f55a20de740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55981bdd90 con 0x56233a40cb40 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f55a20de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55981bcff0 con 0x56233a40b5d0 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f55a20de740 10 client.0 _ll_register_callbacks cb 0x7f558c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559daa1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 0x56233a41e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559e2a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a416370 0x56233a40f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559e2a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a416370 0x56233a40f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39438/0 (socket says 172.21.15.31:39438) 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559e2a2640 1 -- 172.21.15.31:0/1266937859 learned_addr learned my addr 172.21.15.31:0/1266937859 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559d2a0640 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a41ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559e2a2640 1 -- 172.21.15.31:0/1266937859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 msgr2=0x56233a41e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559e2a2640 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 0x56233a41e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559e2a2640 1 -- 172.21.15.31:0/1266937859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 msgr2=0x56233a41ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559e2a2640 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a41ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.567+0000 7f559e2a2640 1 -- 172.21.15.31:0/1266937859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233a40fea0 con 0x56233a416370 2024-09-16T00:43:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.568+0000 7f559e2a2640 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a416370 0x56233a40f830 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f559000f790 tx=0x7f5590005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a4a320e912cf8f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.568+0000 7f55817fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.568+0000 7f55817fa640 1 -- 172.21.15.31:0/1266937859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559000f9a0 con 0x56233a416370 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.568+0000 7f55817fa640 1 -- 172.21.15.31:0/1266937859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f559001e040 con 0x56233a416370 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.568+0000 7f55a20de740 1 -- 172.21.15.31:0/1266937859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56233a378d70 con 0x56233a416370 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.568+0000 7f55817fa640 1 -- 172.21.15.31:0/1266937859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5590034050 con 0x56233a416370 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.568+0000 7f55a20de740 10 client.15603 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.568+0000 7f55a20de740 1 -- 172.21.15.31:0/1266937859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56233a490720 con 0x56233a416370 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.568+0000 7f55817fa640 1 -- 172.21.15.31:0/1266937859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56233a378d70 con 0x56233a416370 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f55817fa640 1 -- 172.21.15.31:0/1266937859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f5590024040 con 0x56233a416370 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f55a20de740 10 client.15603 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f55a20de740 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f55a20de740 20 client.15603 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f55a20de740 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56233a490c70 0x56233a493090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f55a20de740 4 client.15603 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:43:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f55a20de740 1 -- 172.21.15.31:0/1266937859 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56233a2f5170 con 0x56233a490c70 2024-09-16T00:43:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f559daa1640 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56233a490c70 0x56233a493090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f55817fa640 10 client.15603 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.569+0000 7f559daa1640 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56233a490c70 0x56233a493090 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1184168027b2549e server_cookie=ae7118a0cc4e41aa in_seq=0 out_seq=0 2024-09-16T00:43:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.571+0000 7f55817fa640 1 -- 172.21.15.31:0/1266937859 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56233a2f5170 con 0x56233a490c70 2024-09-16T00:43:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.571+0000 7f55817fa640 10 client.15603 handle_command_reply: tid=0 2024-09-16T00:43:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.571+0000 7f55a20de740 1 client.15603 shutdown 2024-09-16T00:43:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.571+0000 7f55a20de740 1 -- 172.21.15.31:0/1266937859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a416370 msgr2=0x56233a40f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.571+0000 7f55a20de740 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a416370 0x56233a40f830 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f559000f790 tx=0x7f5590005c80 comp rx=0 tx=0).stop 2024-09-16T00:43:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55a20de740 1 -- 172.21.15.31:0/1266937859 shutdown_connections 2024-09-16T00:43:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55a20de740 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56233a416370 0x56233a40f830 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55a20de740 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56233a40cb40 0x56233a41ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55a20de740 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56233a490c70 0x56233a493090 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55a20de740 1 --2- 172.21.15.31:0/1266937859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56233a40b5d0 0x56233a41e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55a20de740 1 -- 172.21.15.31:0/1266937859 >> 172.21.15.31:0/1266937859 conn(0x56233a3d4310 msgr2=0x56233a414890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55817fa640 0 client.15603 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55a20de740 1 -- 172.21.15.31:0/1266937859 shutdown_connections 2024-09-16T00:43:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55a20de740 1 -- 172.21.15.31:0/1266937859 wait complete. 2024-09-16T00:43:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f55a20de740 20 client.15603 trim_cache size 0 max 16384 2024-09-16T00:43:22.575 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:43:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f559fd05640 1 -- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55840a3410 msgr2=0x7f55840a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.572+0000 7f559fd05640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55840a3410 0x7f55840a58d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56233a408ac0 tx=0x7f5594005f90 comp rx=0 tx=0).stop 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 -- 172.21.15.31:0/3738301049 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5598074eb0 msgr2=0x7f559810b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5598074eb0 0x7f559810b240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f558c005940 tx=0x7f558c0040c0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 -- 172.21.15.31:0/3738301049 shutdown_connections 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5598112b80 0x7f559810b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5598074eb0 0x7f559810b240 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f55840a3410 0x7f55840a58d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 --2- 172.21.15.31:0/3738301049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5598074590 0x7f55981121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 -- 172.21.15.31:0/3738301049 >> 172.21.15.31:0/3738301049 conn(0x7f55980fd8b0 msgr2=0x7f55980ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 -- 172.21.15.31:0/3738301049 shutdown_connections 2024-09-16T00:43:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:22.573+0000 7f559fd05640 1 -- 172.21.15.31:0/3738301049 wait complete. 2024-09-16T00:43:23.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:23 smithi031 ceph-mon[30956]: pgmap v423: 129 pgs: 129 active+clean; 184 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-16T00:43:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:23 smithi081 ceph-mon[35539]: pgmap v423: 129 pgs: 129 active+clean; 184 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-16T00:43:23.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:23 smithi102 ceph-mon[35364]: pgmap v423: 129 pgs: 129 active+clean; 184 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-16T00:43:24.398 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:43:24.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:43:24.669 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:25.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.179+0000 7f26862ed640 1 -- 172.21.15.31:0/817576117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680101de0 msgr2=0x7f26801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:25.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.179+0000 7f26862ed640 1 --2- 172.21.15.31:0/817576117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680101de0 0x7f26801021c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2670009900 tx=0x7f267002a410 comp rx=0 tx=0).stop 2024-09-16T00:43:25.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.180+0000 7f26862ed640 1 -- 172.21.15.31:0/817576117 shutdown_connections 2024-09-16T00:43:25.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.180+0000 7f26862ed640 1 --2- 172.21.15.31:0/817576117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2680106f10 0x7f268010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.180+0000 7f26862ed640 1 --2- 172.21.15.31:0/817576117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2680102790 0x7f26801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.180+0000 7f26862ed640 1 --2- 172.21.15.31:0/817576117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680101de0 0x7f26801021c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.180+0000 7f26862ed640 1 -- 172.21.15.31:0/817576117 >> 172.21.15.31:0/817576117 conn(0x7f26800fd8c0 msgr2=0x7f26800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:25.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.180+0000 7f26862ed640 1 -- 172.21.15.31:0/817576117 shutdown_connections 2024-09-16T00:43:25.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.180+0000 7f26862ed640 1 -- 172.21.15.31:0/817576117 wait complete. 2024-09-16T00:43:25.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.180+0000 7f26862ed640 1 Processor -- start 2024-09-16T00:43:25.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.180+0000 7f26862ed640 1 -- start start 2024-09-16T00:43:25.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f26862ed640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2680101de0 0x7f26801984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f26862ed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680102790 0x7f2680198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f26862ed640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2680106f10 0x7f268019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f26862ed640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2680111f70 con 0x7f2680102790 2024-09-16T00:43:25.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f26862ed640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2680111df0 con 0x7f2680101de0 2024-09-16T00:43:25.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f26862ed640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f26801120f0 con 0x7f2680106f10 2024-09-16T00:43:25.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680102790 0x7f2680198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680102790 0x7f2680198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39458/0 (socket says 172.21.15.31:39458) 2024-09-16T00:43:25.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267f7fe640 1 -- 172.21.15.31:0/3851594695 learned_addr learned my addr 172.21.15.31:0/3851594695 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:25.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267ffff640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2680101de0 0x7f26801984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f268488a640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2680106f10 0x7f268019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267f7fe640 1 -- 172.21.15.31:0/3851594695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2680106f10 msgr2=0x7f268019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:25.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267f7fe640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2680106f10 0x7f268019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267f7fe640 1 -- 172.21.15.31:0/3851594695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2680101de0 msgr2=0x7f26801984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:25.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267f7fe640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2680101de0 0x7f26801984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267f7fe640 1 -- 172.21.15.31:0/3851594695 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268019d480 con 0x7f2680102790 2024-09-16T00:43:25.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f268488a640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2680106f10 0x7f268019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:25.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.181+0000 7f267f7fe640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680102790 0x7f2680198a10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f267400e9f0 tx=0x7f267400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:25.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.182+0000 7f267d7fa640 1 -- 172.21.15.31:0/3851594695 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267400cd80 con 0x7f2680102790 2024-09-16T00:43:25.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.182+0000 7f267d7fa640 1 -- 172.21.15.31:0/3851594695 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2674010910 con 0x7f2680102790 2024-09-16T00:43:25.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.182+0000 7f26862ed640 1 -- 172.21.15.31:0/3851594695 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268019d770 con 0x7f2680102790 2024-09-16T00:43:25.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.182+0000 7f267d7fa640 1 -- 172.21.15.31:0/3851594695 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674018990 con 0x7f2680102790 2024-09-16T00:43:25.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.182+0000 7f26862ed640 1 -- 172.21.15.31:0/3851594695 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26801a5050 con 0x7f2680102790 2024-09-16T00:43:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.184+0000 7f267d7fa640 1 -- 172.21.15.31:0/3851594695 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2674010ac0 con 0x7f2680102790 2024-09-16T00:43:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.184+0000 7f26886c6740 1 Processor -- start 2024-09-16T00:43:25.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f26886c6740 1 -- start start 2024-09-16T00:43:25.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f26886c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c5524b60 0x55f9c5524f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f26886c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 0x55f9c552f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f26886c6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c552fd40 0x55f9c5532130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f26886c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f9c548a280 con 0x55f9c5524b60 2024-09-16T00:43:25.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f26886c6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5431b10 con 0x55f9c552fd40 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f26886c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5486240 con 0x55f9c5525480 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f268488a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c5524b60 0x55f9c5524f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f268488a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c5524b60 0x55f9c5524f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39462/0 (socket says 172.21.15.31:39462) 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f267f7fe640 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c552fd40 0x55f9c5532130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f268488a640 1 -- 172.21.15.31:0/2156325676 learned_addr learned my addr 172.21.15.31:0/2156325676 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f267d7fa640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26680a3240 0x7f26680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f267ffff640 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 0x55f9c552f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f268488a640 1 -- 172.21.15.31:0/2156325676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 msgr2=0x55f9c552f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f268488a640 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 0x55f9c552f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f268488a640 1 -- 172.21.15.31:0/2156325676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c552fd40 msgr2=0x55f9c5532130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f268488a640 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c552fd40 0x55f9c5532130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f268488a640 1 -- 172.21.15.31:0/2156325676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c543e1f0 con 0x55f9c5524b60 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.185+0000 7f267ffff640 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 0x55f9c552f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.186+0000 7f267d7fa640 1 -- 172.21.15.31:0/3851594695 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f26740d00e0 con 0x7f2680102790 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.186+0000 7f268488a640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26680a3240 0x7f26680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.186+0000 7f268488a640 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c5524b60 0x55f9c5524f40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f267800f3e0 tx=0x7f267800f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2057c8f645685d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.186+0000 7f26667fc640 1 -- 172.21.15.31:0/2156325676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267801c400 con 0x55f9c5524b60 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.186+0000 7f26667fc640 1 -- 172.21.15.31:0/2156325676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267801ca80 con 0x55f9c5524b60 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.186+0000 7f26667fc640 1 -- 172.21.15.31:0/2156325676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780107b0 con 0x55f9c5524b60 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.186+0000 7f268488a640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26680a3240 0x7f26680a5700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2678020000 tx=0x7f267801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 -- 172.21.15.31:0/2156325676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c5524b60 msgr2=0x55f9c5524f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c5524b60 0x55f9c5524f40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f267800f3e0 tx=0x7f267800f7e0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 -- 172.21.15.31:0/2156325676 shutdown_connections 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c552fd40 0x55f9c5532130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 0x55f9c552f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 --2- 172.21.15.31:0/2156325676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c5524b60 0x55f9c5524f40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 -- 172.21.15.31:0/2156325676 >> 172.21.15.31:0/2156325676 conn(0x55f9c55210f0 msgr2=0x55f9c55214d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 -- 172.21.15.31:0/2156325676 shutdown_connections 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 -- 172.21.15.31:0/2156325676 wait complete. 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.187+0000 7f26886c6740 1 -- start start 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f26886c6740 1 Processor -- start 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f26886c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 0x55f9c552cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f26886c6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c552d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f26886c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f26886c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5536d20 con 0x55f9c552fd40 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f26886c6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5444f10 con 0x55f9c5525480 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f26886c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5536ea0 con 0x55f9c5524b60 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f26886c6740 10 client.0 _ll_register_callbacks cb 0x55f9c540da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f268488a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f267f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c552d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f268488a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39474/0 (socket says 172.21.15.31:39474) 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f267f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c552d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38554/0 (socket says 172.21.15.31:38554) 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f268488a640 1 -- 172.21.15.31:0/1846286795 learned_addr learned my addr 172.21.15.31:0/1846286795 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f267ffff640 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 0x55f9c552cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.188+0000 7f268488a640 1 -- 172.21.15.31:0/1846286795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 msgr2=0x55f9c552cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f268488a640 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 0x55f9c552cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f268488a640 1 -- 172.21.15.31:0/1846286795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 msgr2=0x55f9c552d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f268488a640 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c552d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f268488a640 1 -- 172.21.15.31:0/1846286795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c51fb6a0 con 0x55f9c552fd40 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f267f7fe640 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c552d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f268488a640 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552d630 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f267801cef0 tx=0x7f2678012570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652f2c2c633fbdc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f265f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f265f7fe640 1 -- 172.21.15.31:0/1846286795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678033060 con 0x55f9c552fd40 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f26886c6740 1 -- 172.21.15.31:0/1846286795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267801ca80 con 0x55f9c552fd40 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f265f7fe640 1 -- 172.21.15.31:0/1846286795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f2678010a10 con 0x55f9c552fd40 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f265f7fe640 1 -- 172.21.15.31:0/1846286795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f267803a050 con 0x55f9c552fd40 2024-09-16T00:43:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f26886c6740 10 client.15615 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f26886c6740 1 -- 172.21.15.31:0/1846286795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f9c55428b0 con 0x55f9c552fd40 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.189+0000 7f265f7fe640 1 -- 172.21.15.31:0/1846286795 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267801ca80 con 0x55f9c552fd40 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f265f7fe640 1 -- 172.21.15.31:0/1846286795 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f2678003af0 con 0x55f9c552fd40 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f26886c6740 10 client.15615 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f26886c6740 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f26886c6740 20 client.15615 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f26886c6740 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9c5543d10 0x55f9c5546100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f26886c6740 4 client.15615 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f26886c6740 1 -- 172.21.15.31:0/1846286795 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9c54977f0 con 0x55f9c5543d10 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f267ffff640 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9c5543d10 0x55f9c5546100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f267ffff640 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9c5543d10 0x55f9c5546100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451218dab9f44c1b server_cookie=5c2ee397a405f718 in_seq=0 out_seq=0 2024-09-16T00:43:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.190+0000 7f265f7fe640 10 client.15615 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f265f7fe640 1 -- 172.21.15.31:0/1846286795 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9c54977f0 con 0x55f9c5543d10 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f265f7fe640 10 client.15615 handle_command_reply: tid=0 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 client.15615 shutdown 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 -- 172.21.15.31:0/1846286795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 msgr2=0x55f9c552d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552d630 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f267801cef0 tx=0x7f2678012570 comp rx=0 tx=0).stop 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 -- 172.21.15.31:0/1846286795 shutdown_connections 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552d630 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c552d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9c5543d10 0x55f9c5546100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 --2- 172.21.15.31:0/1846286795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 0x55f9c552cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f265f7fe640 0 client.15615 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 -- 172.21.15.31:0/1846286795 >> 172.21.15.31:0/1846286795 conn(0x55f9c55210f0 msgr2=0x55f9c5520270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 -- 172.21.15.31:0/1846286795 shutdown_connections 2024-09-16T00:43:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 1 -- 172.21.15.31:0/1846286795 wait complete. 2024-09-16T00:43:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.193+0000 7f26886c6740 20 client.15615 trim_cache size 0 max 16384 2024-09-16T00:43:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f26886c6740 1 Processor -- start 2024-09-16T00:43:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f26886c6740 1 -- start start 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f26886c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 0x55f9c554b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f26886c6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c554bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f26886c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f26886c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f26801aab40 con 0x55f9c552fd40 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f26886c6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f26801ad3c0 con 0x55f9c5525480 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f26886c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f26801ab1b0 con 0x55f9c5524b60 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f268488a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f268488a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39478/0 (socket says 172.21.15.31:39478) 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f267f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c554bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f268488a640 1 -- 172.21.15.31:0/2246513963 learned_addr learned my addr 172.21.15.31:0/2246513963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.214+0000 7f267ffff640 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 0x55f9c554b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f268488a640 1 -- 172.21.15.31:0/2246513963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 msgr2=0x55f9c554b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f268488a640 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 0x55f9c554b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f268488a640 1 -- 172.21.15.31:0/2246513963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 msgr2=0x55f9c554bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f268488a640 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c554bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f268488a640 1 -- 172.21.15.31:0/2246513963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c551c830 con 0x55f9c552fd40 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f267ffff640 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 0x55f9c554b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f267f7fe640 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c554bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f268488a640 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552cb70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2678000f80 tx=0x7f2678009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197c32246b8f87d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f2665ffb640 1 -- 172.21.15.31:0/2246513963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678033040 con 0x55f9c552fd40 2024-09-16T00:43:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f2665ffb640 1 -- 172.21.15.31:0/2246513963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267802a040 con 0x55f9c552fd40 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.215+0000 7f2665ffb640 1 -- 172.21.15.31:0/2246513963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780048c0 con 0x55f9c552fd40 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 -- 172.21.15.31:0/2246513963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 msgr2=0x55f9c552cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552cb70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2678000f80 tx=0x7f2678009cd0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 -- 172.21.15.31:0/2246513963 shutdown_connections 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c552cb70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5525480 0x55f9c554bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 --2- 172.21.15.31:0/2246513963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5524b60 0x55f9c554b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 -- 172.21.15.31:0/2246513963 >> 172.21.15.31:0/2246513963 conn(0x55f9c55210f0 msgr2=0x55f9c552eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 -- 172.21.15.31:0/2246513963 shutdown_connections 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 -- 172.21.15.31:0/2246513963 wait complete. 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 -- start start 2024-09-16T00:43:25.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.216+0000 7f26886c6740 1 Processor -- start 2024-09-16T00:43:25.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f26886c6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5524b60 0x55f9c5537e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f26886c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 0x55f9c5538390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f26886c6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c55388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f26886c6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f26801ab880 con 0x55f9c552fd40 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f26886c6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f26801accf0 con 0x55f9c5524b60 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f26886c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f26801abf50 con 0x55f9c5525480 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f26886c6740 10 client.0 _ll_register_callbacks cb 0x7f267801f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f268488a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c55388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5524b60 0x55f9c5537e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5524b60 0x55f9c5537e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38578/0 (socket says 172.21.15.31:38578) 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f268488a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c55388d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39492/0 (socket says 172.21.15.31:39492) 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267ffff640 1 -- 172.21.15.31:0/1587791697 learned_addr learned my addr 172.21.15.31:0/1587791697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267f7fe640 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 0x55f9c5538390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267ffff640 1 -- 172.21.15.31:0/1587791697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 msgr2=0x55f9c5538390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267ffff640 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 0x55f9c5538390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267ffff640 1 -- 172.21.15.31:0/1587791697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 msgr2=0x55f9c55388d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267ffff640 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c55388d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267ffff640 1 -- 172.21.15.31:0/1587791697 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c55ad840 con 0x55f9c5524b60 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f268488a640 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c55388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f267ffff640 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5524b60 0x55f9c5537e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2670009900 tx=0x7f2670031920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c2640f0764a9804 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.217+0000 7f265f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f265f7fe640 1 -- 172.21.15.31:0/1587791697 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670035040 con 0x55f9c5524b60 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f265f7fe640 1 -- 172.21.15.31:0/1587791697 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f2670004450 con 0x55f9c5524b60 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f26886c6740 1 -- 172.21.15.31:0/1587791697 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267802a040 con 0x55f9c5524b60 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f265f7fe640 1 -- 172.21.15.31:0/1587791697 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2670055050 con 0x55f9c5524b60 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f26886c6740 10 client.25307 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f26886c6740 1 -- 172.21.15.31:0/1587791697 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f9c55aeb30 con 0x55f9c5524b60 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f265f7fe640 1 -- 172.21.15.31:0/1587791697 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267802a040 con 0x55f9c5524b60 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f265f7fe640 1 -- 172.21.15.31:0/1587791697 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f2670047700 con 0x55f9c5524b60 2024-09-16T00:43:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f26886c6740 10 client.25307 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f26886c6740 10 client.25307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f26886c6740 20 client.25307 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f26886c6740 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9c55af2b0 0x55f9c55b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f26886c6740 4 client.25307 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.218+0000 7f26886c6740 1 -- 172.21.15.31:0/1587791697 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f9c5491d00 con 0x55f9c55af2b0 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.219+0000 7f267f7fe640 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9c55af2b0 0x55f9c55b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.219+0000 7f267f7fe640 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9c55af2b0 0x55f9c55b16d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2100c00d6c7ecfe8 server_cookie=5d67d7bfa36d03e9 in_seq=0 out_seq=0 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.219+0000 7f265f7fe640 10 client.25307 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f265f7fe640 1 -- 172.21.15.31:0/1587791697 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f9c5491d00 con 0x55f9c55af2b0 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f265f7fe640 10 client.25307 handle_command_reply: tid=0 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 client.25307 shutdown 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 -- 172.21.15.31:0/1587791697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5524b60 msgr2=0x55f9c5537e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5524b60 0x55f9c5537e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2670009900 tx=0x7f2670031920 comp rx=0 tx=0).stop 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 -- 172.21.15.31:0/1587791697 shutdown_connections 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9c552fd40 0x55f9c55388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9c55af2b0 0x55f9c55b16d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9c5525480 0x55f9c5538390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 --2- 172.21.15.31:0/1587791697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9c5524b60 0x55f9c5537e50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 -- 172.21.15.31:0/1587791697 >> 172.21.15.31:0/1587791697 conn(0x55f9c55210f0 msgr2=0x55f9c551fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f265f7fe640 0 client.25307 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 -- 172.21.15.31:0/1587791697 shutdown_connections 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.221+0000 7f26886c6740 1 -- 172.21.15.31:0/1587791697 wait complete. 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26886c6740 20 client.25307 trim_cache size 0 max 16384 2024-09-16T00:43:25.226 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:43:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:43:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "46b95026-70da-4b99-a39e-cf3a45db4b32", 2024-09-16T00:43:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:43:25.227 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:43:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 -- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26680a3240 msgr2=0x7f26680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26680a3240 0x7f26680a5700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2678020000 tx=0x7f267801d000 comp rx=0 tx=0).stop 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 -- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680102790 msgr2=0x7f2680198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680102790 0x7f2680198a10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f267400e9f0 tx=0x7f267400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 -- 172.21.15.31:0/3851594695 shutdown_connections 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f26680a3240 0x7f26680a5700 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2680106f10 0x7f268019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2680102790 0x7f2680198a10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 --2- 172.21.15.31:0/3851594695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2680101de0 0x7f26801984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 -- 172.21.15.31:0/3851594695 >> 172.21.15.31:0/3851594695 conn(0x7f26800fd8c0 msgr2=0x7f26800fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 -- 172.21.15.31:0/3851594695 shutdown_connections 2024-09-16T00:43:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:25.222+0000 7f26862ed640 1 -- 172.21.15.31:0/3851594695 wait complete. 2024-09-16T00:43:25.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:25 smithi031 ceph-mon[30956]: pgmap v424: 129 pgs: 129 active+clean; 189 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 135 op/s 2024-09-16T00:43:25.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:25 smithi081 ceph-mon[35539]: pgmap v424: 129 pgs: 129 active+clean; 189 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 135 op/s 2024-09-16T00:43:25.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:25 smithi102 ceph-mon[35364]: pgmap v424: 129 pgs: 129 active+clean; 189 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 135 op/s 2024-09-16T00:43:25.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:43:26.186 INFO:tasks.workunit.client.0.smithi031.stdout: 47 559732 372 339150 216 298495 593 2024-09-16T00:43:26.263 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:26.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:26 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:43:26.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:26 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:26.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:26 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:43:26.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:26 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:43:26.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:26 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:26.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:26 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:43:26.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 -- 172.21.15.31:0/4142044862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac100370 msgr2=0x7f28ac100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:26.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 --2- 172.21.15.31:0/4142044862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac100370 0x7f28ac100750 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f289c009930 tx=0x7f289c02a420 comp rx=0 tx=0).stop 2024-09-16T00:43:26.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 -- 172.21.15.31:0/4142044862 shutdown_connections 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 --2- 172.21.15.31:0/4142044862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28ac106fc0 0x7f28ac10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 --2- 172.21.15.31:0/4142044862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28ac100d20 0x7f28ac106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 --2- 172.21.15.31:0/4142044862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac100370 0x7f28ac100750 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 -- 172.21.15.31:0/4142044862 >> 172.21.15.31:0/4142044862 conn(0x7f28ac077210 msgr2=0x7f28ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 -- 172.21.15.31:0/4142044862 shutdown_connections 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 -- 172.21.15.31:0/4142044862 wait complete. 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 Processor -- start 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.748+0000 7f28b2274640 1 -- start start 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28b2274640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac100370 0x7f28ac198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28b2274640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28ac100d20 0x7f28ac198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28b2274640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28ac106fc0 0x7f28ac19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28b2274640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac111fa0 con 0x7f28ac100370 2024-09-16T00:43:26.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28b2274640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac111e20 con 0x7f28ac100d20 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28b2274640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac112120 con 0x7f28ac106fc0 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28ab7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28ac100d20 0x7f28ac198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac100370 0x7f28ac198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28b0811640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28ac106fc0 0x7f28ac19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28ab7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28ac100d20 0x7f28ac198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38592/0 (socket says 172.21.15.31:38592) 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac100370 0x7f28ac198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39518/0 (socket says 172.21.15.31:39518) 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28b0811640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28ac106fc0 0x7f28ac19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:60562/0 (socket says 172.21.15.31:60562) 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28ab7fe640 1 -- 172.21.15.31:0/496017136 learned_addr learned my addr 172.21.15.31:0/496017136 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28ab7fe640 1 -- 172.21.15.31:0/496017136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28ac106fc0 msgr2=0x7f28ac19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28ac106fc0 0x7f28ac19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28ab7fe640 1 -- 172.21.15.31:0/496017136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac100370 msgr2=0x7f28ac198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac100370 0x7f28ac198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28ab7fe640 1 -- 172.21.15.31:0/496017136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac19d4d0 con 0x7f28ac100d20 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.749+0000 7f28b0811640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28ac106fc0 0x7f28ac19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.750+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28ac100d20 0x7f28ac198a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f28a000e9f0 tx=0x7f28a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.750+0000 7f28a97fa640 1 -- 172.21.15.31:0/496017136 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a000ccc0 con 0x7f28ac100d20 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.750+0000 7f28a97fa640 1 -- 172.21.15.31:0/496017136 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a0005500 con 0x7f28ac100d20 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.750+0000 7f28b2274640 1 -- 172.21.15.31:0/496017136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ac19d7c0 con 0x7f28ac100d20 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.750+0000 7f28a97fa640 1 -- 172.21.15.31:0/496017136 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0010690 con 0x7f28ac100d20 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.750+0000 7f28b2274640 1 -- 172.21.15.31:0/496017136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ac1a50a0 con 0x7f28ac100d20 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28a97fa640 1 -- 172.21.15.31:0/496017136 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f28a00046c0 con 0x7f28ac100d20 2024-09-16T00:43:26.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28b464d740 1 Processor -- start 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28b464d740 1 -- start start 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28b464d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 0x55608175a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28b464d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55608175ac80 0x556081769550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28b464d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556081769a90 0x55608176be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28b464d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5560816c02e0 con 0x55608175ac80 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28b464d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556081667ce0 con 0x556081769a90 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28b464d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5560816bc2a0 con 0x55608175a360 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28a97fa640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28940a3240 0x7f28940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55608175ac80 0x556081769550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55608175ac80 0x556081769550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39526/0 (socket says 172.21.15.31:39526) 2024-09-16T00:43:26.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28abfff640 1 -- 172.21.15.31:0/599822233 learned_addr learned my addr 172.21.15.31:0/599822233 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.753+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/599822233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556081769a90 0x55608176be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28b0811640 1 --2- 172.21.15.31:0/599822233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 0x55608175a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28b0811640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28940a3240 0x7f28940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28abfff640 1 -- 172.21.15.31:0/599822233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 msgr2=0x55608175a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28abfff640 1 --2- 172.21.15.31:0/599822233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 0x55608175a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28abfff640 1 -- 172.21.15.31:0/599822233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556081769a90 msgr2=0x55608176be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28abfff640 1 --2- 172.21.15.31:0/599822233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556081769a90 0x55608176be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28abfff640 1 -- 172.21.15.31:0/599822233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55608164c440 con 0x55608175ac80 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28a97fa640 1 -- 172.21.15.31:0/496017136 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f28a00d0050 con 0x7f28ac100d20 2024-09-16T00:43:26.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28abfff640 1 --2- 172.21.15.31:0/599822233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55608175ac80 0x556081769550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f289c02a940 tx=0x7f289c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ac65d65df3f160 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28b0811640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28940a3240 0x7f28940a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f28a40099e0 tx=0x7f28a4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.754+0000 7f28927fc640 1 -- 172.21.15.31:0/599822233 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c040070 con 0x55608175ac80 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28927fc640 1 -- 172.21.15.31:0/599822233 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c031870 con 0x55608175ac80 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28927fc640 1 -- 172.21.15.31:0/599822233 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c035040 con 0x55608175ac80 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28b464d740 1 -- 172.21.15.31:0/599822233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55608175ac80 msgr2=0x556081769550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28b464d740 1 --2- 172.21.15.31:0/599822233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55608175ac80 0x556081769550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f289c02a940 tx=0x7f289c005bd0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28b464d740 1 -- 172.21.15.31:0/599822233 shutdown_connections 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28b464d740 1 --2- 172.21.15.31:0/599822233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556081769a90 0x55608176be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28b464d740 1 --2- 172.21.15.31:0/599822233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55608175ac80 0x556081769550 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28b464d740 1 --2- 172.21.15.31:0/599822233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 0x55608175a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28b464d740 1 -- 172.21.15.31:0/599822233 >> 172.21.15.31:0/599822233 conn(0x556081756a30 msgr2=0x556081756e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.755+0000 7f28b464d740 1 -- 172.21.15.31:0/599822233 shutdown_connections 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.756+0000 7f28b464d740 1 -- 172.21.15.31:0/599822233 wait complete. 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.756+0000 7f28b464d740 1 -- start start 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.756+0000 7f28b464d740 1 Processor -- start 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28b464d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28b464d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 0x556081766660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28b464d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x556081766ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28b464d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55608176c600 con 0x556081769a90 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28b464d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55608167b050 con 0x55608175a360 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28b464d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55608176c780 con 0x55608175ac80 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28b464d740 10 client.0 _ll_register_callbacks cb 0x5560816bb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28abfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28abfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:38608/0 (socket says 172.21.15.31:38608) 2024-09-16T00:43:26.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28b0811640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x556081766ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28abfff640 1 -- 172.21.15.31:0/2297068921 learned_addr learned my addr 172.21.15.31:0/2297068921 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 0x556081766660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28abfff640 1 -- 172.21.15.31:0/2297068921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 msgr2=0x556081766660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28abfff640 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 0x556081766660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28abfff640 1 -- 172.21.15.31:0/2297068921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 msgr2=0x556081766ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28abfff640 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x556081766ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28abfff640 1 -- 172.21.15.31:0/2297068921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560814313c0 con 0x55608175a360 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28b0811640 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x556081766ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f28abfff640 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817693b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f289c009a50 tx=0x7f289c02a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9c35e27ebbe197 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.757+0000 7f288b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f288b7fe640 1 -- 172.21.15.31:0/2297068921 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c04c020 con 0x55608175a360 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28b464d740 1 -- 172.21.15.31:0/2297068921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289c031870 con 0x55608175a360 2024-09-16T00:43:26.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f288b7fe640 1 -- 172.21.15.31:0/2297068921 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f289c004980 con 0x55608175a360 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f288b7fe640 1 -- 172.21.15.31:0/2297068921 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f289c057050 con 0x55608175a360 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28b464d740 10 client.25319 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28b464d740 1 -- 172.21.15.31:0/2297068921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556081777fc0 con 0x55608175a360 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f288b7fe640 1 -- 172.21.15.31:0/2297068921 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c031870 con 0x55608175a360 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f288b7fe640 1 -- 172.21.15.31:0/2297068921 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f289c03ca40 con 0x55608175a360 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28b464d740 10 client.25319 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28b464d740 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28b464d740 20 client.25319 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28b464d740 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556081779310 0x55608177b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28b464d740 4 client.25319 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28b464d740 1 -- 172.21.15.31:0/2297068921 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560816cd850 con 0x556081779310 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.758+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556081779310 0x55608177b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.759+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556081779310 0x55608177b700 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28b9d106dbe5123 server_cookie=afdcb0c7c4443365 in_seq=0 out_seq=0 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.759+0000 7f288b7fe640 10 client.25319 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f288b7fe640 1 -- 172.21.15.31:0/2297068921 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560816cd850 con 0x556081779310 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f288b7fe640 10 client.25319 handle_command_reply: tid=0 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 client.25319 shutdown 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 -- 172.21.15.31:0/2297068921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 msgr2=0x5560817693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817693b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f289c009a50 tx=0x7f289c02a420 comp rx=0 tx=0).stop 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 -- 172.21.15.31:0/2297068921 shutdown_connections 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x556081766ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556081779310 0x55608177b700 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 0x556081766660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 --2- 172.21.15.31:0/2297068921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817693b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f288b7fe640 0 client.25319 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:26.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 -- 172.21.15.31:0/2297068921 >> 172.21.15.31:0/2297068921 conn(0x556081756a30 msgr2=0x55608175c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:26.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.761+0000 7f28b464d740 1 -- 172.21.15.31:0/2297068921 shutdown_connections 2024-09-16T00:43:26.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.762+0000 7f28b464d740 1 -- 172.21.15.31:0/2297068921 wait complete. 2024-09-16T00:43:26.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.762+0000 7f28b464d740 20 client.25319 trim_cache size 0 max 16384 2024-09-16T00:43:26.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b464d740 1 Processor -- start 2024-09-16T00:43:26.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b464d740 1 -- start start 2024-09-16T00:43:26.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b464d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b464d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 0x556081780ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b464d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x5560817811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b464d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1aab30 con 0x556081769a90 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b464d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1ad3b0 con 0x55608175a360 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b464d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1ab1a0 con 0x55608175ac80 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b0811640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x5560817811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b0811640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x5560817811e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39532/0 (socket says 172.21.15.31:39532) 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b0811640 1 -- 172.21.15.31:0/1263194275 learned_addr learned my addr 172.21.15.31:0/1263194275 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28abfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/1263194275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 0x556081780ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b0811640 1 -- 172.21.15.31:0/1263194275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 msgr2=0x556081780ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b0811640 1 --2- 172.21.15.31:0/1263194275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 0x556081780ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b0811640 1 -- 172.21.15.31:0/1263194275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 msgr2=0x5560817758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b0811640 1 --2- 172.21.15.31:0/1263194275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.783+0000 7f28b0811640 1 -- 172.21.15.31:0/1263194275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556081764bb0 con 0x556081769a90 2024-09-16T00:43:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.784+0000 7f28b0811640 1 --2- 172.21.15.31:0/1263194275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x5560817811e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f28a400fe30 tx=0x7f28a401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8491e76efbd21be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.784+0000 7f2891ffb640 1 -- 172.21.15.31:0/1263194275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a401fdd0 con 0x556081769a90 2024-09-16T00:43:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.784+0000 7f2891ffb640 1 -- 172.21.15.31:0/1263194275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a401f0d0 con 0x556081769a90 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.784+0000 7f2891ffb640 1 -- 172.21.15.31:0/1263194275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4010c50 con 0x556081769a90 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.785+0000 7f28b464d740 1 -- 172.21.15.31:0/1263194275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 msgr2=0x5560817811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.785+0000 7f28b464d740 1 --2- 172.21.15.31:0/1263194275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x5560817811e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f28a400fe30 tx=0x7f28a401a000 comp rx=0 tx=0).stop 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.785+0000 7f28b464d740 1 -- 172.21.15.31:0/1263194275 shutdown_connections 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.785+0000 7f28b464d740 1 --2- 172.21.15.31:0/1263194275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x5560817811e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.785+0000 7f28b464d740 1 --2- 172.21.15.31:0/1263194275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175ac80 0x556081780ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.785+0000 7f28b464d740 1 --2- 172.21.15.31:0/1263194275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175a360 0x5560817758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.785+0000 7f28b464d740 1 -- 172.21.15.31:0/1263194275 >> 172.21.15.31:0/1263194275 conn(0x556081756a30 msgr2=0x556081761d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.785+0000 7f28b464d740 1 -- 172.21.15.31:0/1263194275 shutdown_connections 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.786+0000 7f28b464d740 1 -- 172.21.15.31:0/1263194275 wait complete. 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.786+0000 7f28b464d740 1 -- start start 2024-09-16T00:43:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.786+0000 7f28b464d740 1 Processor -- start 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b464d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 0x556081771ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b464d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175ac80 0x556081772020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b464d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x55608176cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b464d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1ab870 con 0x556081769a90 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b464d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1acce0 con 0x55608175ac80 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b464d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1abf40 con 0x55608175a360 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b0811640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x55608176cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28abfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 0x556081771ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b0811640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x55608176cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39542/0 (socket says 172.21.15.31:39542) 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b0811640 1 -- 172.21.15.31:0/1385311127 learned_addr learned my addr 172.21.15.31:0/1385311127 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175ac80 0x556081772020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28b464d740 10 client.0 _ll_register_callbacks cb 0x7f28a009c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:26.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28abfff640 1 -- 172.21.15.31:0/1385311127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175ac80 msgr2=0x556081772020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28abfff640 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175ac80 0x556081772020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28abfff640 1 -- 172.21.15.31:0/1385311127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 msgr2=0x55608176cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28abfff640 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x55608176cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28abfff640 1 -- 172.21.15.31:0/1385311127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55608176d7a0 con 0x55608175a360 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f28abfff640 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 0x556081771ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f289c001eb0 tx=0x7f289c005550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58435b9f7b5c7bbd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.787+0000 7f288b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f288b7fe640 1 -- 172.21.15.31:0/1385311127 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c04c040 con 0x55608175a360 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f288b7fe640 1 -- 172.21.15.31:0/1385311127 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f289c031870 con 0x55608175a360 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28b464d740 1 -- 172.21.15.31:0/1385311127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28a401f0d0 con 0x55608175a360 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f288b7fe640 1 -- 172.21.15.31:0/1385311127 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f289c040070 con 0x55608175a360 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28b464d740 10 client.25273 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28b464d740 1 -- 172.21.15.31:0/1385311127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5560817e3720 con 0x55608175a360 2024-09-16T00:43:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f288b7fe640 1 -- 172.21.15.31:0/1385311127 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a401f0d0 con 0x55608175a360 2024-09-16T00:43:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f288b7fe640 1 -- 172.21.15.31:0/1385311127 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f289c03c900 con 0x55608175a360 2024-09-16T00:43:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28b464d740 10 client.25273 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28b464d740 10 client.25273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28b464d740 20 client.25273 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28b464d740 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5560817e3e30 0x5560817e6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28b464d740 4 client.25273 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:43:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28b464d740 1 -- 172.21.15.31:0/1385311127 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560816c7d60 con 0x5560817e3e30 2024-09-16T00:43:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.788+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5560817e3e30 0x5560817e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:26.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.789+0000 7f28ab7fe640 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5560817e3e30 0x5560817e6220 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aadc55d223a428c server_cookie=ff90cc7df8836eb4 in_seq=0 out_seq=0 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.789+0000 7f288b7fe640 10 client.25273 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f288b7fe640 1 -- 172.21.15.31:0/1385311127 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5560816c7d60 con 0x5560817e3e30 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f288b7fe640 10 client.25273 handle_command_reply: tid=0 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 client.25273 shutdown 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 -- 172.21.15.31:0/1385311127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 msgr2=0x556081771ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 0x556081771ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f289c001eb0 tx=0x7f289c005550 comp rx=0 tx=0).stop 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 -- 172.21.15.31:0/1385311127 shutdown_connections 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556081769a90 0x55608176cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55608175ac80 0x556081772020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5560817e3e30 0x5560817e6220 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 --2- 172.21.15.31:0/1385311127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55608175a360 0x556081771ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 -- 172.21.15.31:0/1385311127 >> 172.21.15.31:0/1385311127 conn(0x556081756a30 msgr2=0x556081764f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f288b7fe640 0 client.25273 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 -- 172.21.15.31:0/1385311127 shutdown_connections 2024-09-16T00:43:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 1 -- 172.21.15.31:0/1385311127 wait complete. 2024-09-16T00:43:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.791+0000 7f28b464d740 20 client.25273 trim_cache size 0 max 16384 2024-09-16T00:43:26.795 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:43:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (5207 inodes in the stack)", 2024-09-16T00:43:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:43:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "46b95026-70da-4b99-a39e-cf3a45db4b32": { 2024-09-16T00:43:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:43:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "46b95026-70da-4b99-a39e-cf3a45db4b32", 2024-09-16T00:43:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:43:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:26.796 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 -- 172.21.15.31:0/496017136 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28940a3240 msgr2=0x7f28940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28940a3240 0x7f28940a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f28a40099e0 tx=0x7f28a4009500 comp rx=0 tx=0).stop 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 -- 172.21.15.31:0/496017136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28ac100d20 msgr2=0x7f28ac198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28ac100d20 0x7f28ac198a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f28a000e9f0 tx=0x7f28a000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 -- 172.21.15.31:0/496017136 shutdown_connections 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f28940a3240 0x7f28940a5700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28ac106fc0 0x7f28ac19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28ac100d20 0x7f28ac198a60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 --2- 172.21.15.31:0/496017136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ac100370 0x7f28ac198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 -- 172.21.15.31:0/496017136 >> 172.21.15.31:0/496017136 conn(0x7f28ac077210 msgr2=0x7f28ac0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 -- 172.21.15.31:0/496017136 shutdown_connections 2024-09-16T00:43:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:26.792+0000 7f28b2274640 1 -- 172.21.15.31:0/496017136 wait complete. 2024-09-16T00:43:26.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:26 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:43:26.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:26 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:26.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:26 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:43:27.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:27 smithi031 ceph-mon[30956]: pgmap v425: 129 pgs: 129 active+clean; 189 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-09-16T00:43:27.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:27 smithi081 ceph-mon[35539]: pgmap v425: 129 pgs: 129 active+clean; 189 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-09-16T00:43:27.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:27 smithi102 ceph-mon[35364]: pgmap v425: 129 pgs: 129 active+clean; 189 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-09-16T00:43:27.808 INFO:tasks.cephfs.filesystem:scrub status for tag:46b95026-70da-4b99-a39e-cf3a45db4b32 - {'path': '/', 'tag': '46b95026-70da-4b99-a39e-cf3a45db4b32', 'options': 'recursive,force'} 2024-09-16T00:43:29.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:29 smithi031 ceph-mon[30956]: pgmap v426: 129 pgs: 129 active+clean; 195 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 251 op/s 2024-09-16T00:43:29.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:29 smithi081 ceph-mon[35539]: pgmap v426: 129 pgs: 129 active+clean; 195 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 251 op/s 2024-09-16T00:43:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:29 smithi102 ceph-mon[35364]: pgmap v426: 129 pgs: 129 active+clean; 195 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 251 op/s 2024-09-16T00:43:31.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93a77fe640 1 -- 172.21.15.31:0/4047290895 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x555db9017f40 con 0x555db91bdb60 2024-09-16T00:43:31.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93a77fe640 10 client.15579 handle_command_reply: tid=0 2024-09-16T00:43:31.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 client.15579 shutdown 2024-09-16T00:43:31.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 -- 172.21.15.31:0/4047290895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 msgr2=0x555db9142e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db9142e80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f93b801e7f0 tx=0x7f93b80042c0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 -- 172.21.15.31:0/4047290895 shutdown_connections 2024-09-16T00:43:31.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555db913aa30 0x555db913e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555db91bdb60 0x555db91bff80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555db9130170 0x555db913e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 --2- 172.21.15.31:0/4047290895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555db912f850 0x555db9142e80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 -- 172.21.15.31:0/4047290895 >> 172.21.15.31:0/4047290895 conn(0x555db912bde0 msgr2=0x555db91390a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:31.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93a77fe640 0 client.15579 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:31.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 -- 172.21.15.31:0/4047290895 shutdown_connections 2024-09-16T00:43:31.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 1 -- 172.21.15.31:0/4047290895 wait complete. 2024-09-16T00:43:31.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.547+0000 7f93d0863740 20 client.15579 trim_cache size 0 max 16384 2024-09-16T00:43:31.549 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:43:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 6, 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "33404980": { 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 6, 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 139.6, 2024-09-16T00:43:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:31.551 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:43:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.548+0000 7f93ce48a640 1 -- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93b00a3240 msgr2=0x7f93b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.548+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93b00a3240 0x7f93b00a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x555db912cc40 tx=0x7f93b800b390 comp rx=0 tx=0).stop 2024-09-16T00:43:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.548+0000 7f93ce48a640 1 -- 172.21.15.31:0/2208371458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c8102570 msgr2=0x7f93c80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.548+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c8102570 0x7f93c80fd1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93bc005940 tx=0x7f93bc003c10 comp rx=0 tx=0).stop 2024-09-16T00:43:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.549+0000 7f93ce48a640 1 -- 172.21.15.31:0/2208371458 shutdown_connections 2024-09-16T00:43:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.549+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8102ef0 0x7f93c80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.549+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c8102570 0x7f93c80fd1c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.549+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f93b00a3240 0x7f93b00a5700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.549+0000 7f93ce48a640 1 --2- 172.21.15.31:0/2208371458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93c8100e20 0x7f93c80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.549+0000 7f93ce48a640 1 -- 172.21.15.31:0/2208371458 >> 172.21.15.31:0/2208371458 conn(0x7f93c80fc7d0 msgr2=0x7f93c806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:31.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.549+0000 7f93ce48a640 1 -- 172.21.15.31:0/2208371458 shutdown_connections 2024-09-16T00:43:31.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:31.549+0000 7f93ce48a640 1 -- 172.21.15.31:0/2208371458 wait complete. 2024-09-16T00:43:31.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:31 smithi031 ceph-mon[30956]: pgmap v427: 129 pgs: 129 active+clean; 197 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-09-16T00:43:31.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:31 smithi081 ceph-mon[35539]: pgmap v427: 129 pgs: 129 active+clean; 197 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-09-16T00:43:31.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:31 smithi102 ceph-mon[35364]: pgmap v427: 129 pgs: 129 active+clean; 197 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-09-16T00:43:32.529 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '33404980', quiesce took 18.5 seconds. Will release after: 1.0500000000000007 2024-09-16T00:43:33.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:43:33.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:33 smithi031 ceph-mon[30956]: pgmap v428: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-09-16T00:43:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:33 smithi081 ceph-mon[35539]: pgmap v428: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-09-16T00:43:33.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:33 smithi102 ceph-mon[35364]: pgmap v428: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-09-16T00:43:33.855 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:34.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 -- 172.21.15.31:0/2890267762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6281036e0 msgr2=0x7fd628107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 --2- 172.21.15.31:0/2890267762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6281036e0 0x7fd628107b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd610009930 tx=0x7fd61002a420 comp rx=0 tx=0).stop 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 -- 172.21.15.31:0/2890267762 shutdown_connections 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 --2- 172.21.15.31:0/2890267762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6281082a0 0x7fd62810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 --2- 172.21.15.31:0/2890267762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6281036e0 0x7fd628107b90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 --2- 172.21.15.31:0/2890267762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd628102d30 0x7fd628103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 -- 172.21.15.31:0/2890267762 >> 172.21.15.31:0/2890267762 conn(0x7fd6280fc7d0 msgr2=0x7fd6280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 -- 172.21.15.31:0/2890267762 shutdown_connections 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 -- 172.21.15.31:0/2890267762 wait complete. 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 Processor -- start 2024-09-16T00:43:34.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.353+0000 7fd62df9a640 1 -- start start 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd62df9a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd628102d30 0x7fd628100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd62df9a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6281036e0 0x7fd628100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd62df9a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6281082a0 0x7fd628101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd62df9a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281120b0 con 0x7fd628102d30 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd62df9a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd628111f30 con 0x7fd6281082a0 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd62df9a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd628112230 con 0x7fd6281036e0 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd6277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd628102d30 0x7fd628100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd6277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd628102d30 0x7fd628100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38918/0 (socket says 172.21.15.31:38918) 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd61ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6281036e0 0x7fd628100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd6277fe640 1 -- 172.21.15.31:0/457032502 learned_addr learned my addr 172.21.15.31:0/457032502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd627fff640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6281082a0 0x7fd628101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd6277fe640 1 -- 172.21.15.31:0/457032502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6281036e0 msgr2=0x7fd628100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd6277fe640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6281036e0 0x7fd628100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd6277fe640 1 -- 172.21.15.31:0/457032502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6281082a0 msgr2=0x7fd628101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd6277fe640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6281082a0 0x7fd628101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd6277fe640 1 -- 172.21.15.31:0/457032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6281a6a40 con 0x7fd628102d30 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd61ffff640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6281036e0 0x7fd628100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd627fff640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6281082a0 0x7fd628101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.354+0000 7fd6277fe640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd628102d30 0x7fd628100990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9f0 tx=0x7fd61800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.355+0000 7fd6257fa640 1 -- 172.21.15.31:0/457032502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61800cd80 con 0x7fd628102d30 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.355+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6281a6d30 con 0x7fd628102d30 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.355+0000 7fd6257fa640 1 -- 172.21.15.31:0/457032502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd618010910 con 0x7fd628102d30 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.355+0000 7fd6257fa640 1 -- 172.21.15.31:0/457032502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180188f0 con 0x7fd628102d30 2024-09-16T00:43:34.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.355+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281a7230 con 0x7fd628102d30 2024-09-16T00:43:34.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.356+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd628103110 con 0x7fd628102d30 2024-09-16T00:43:34.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.357+0000 7fd6257fa640 1 -- 172.21.15.31:0/457032502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd618010ac0 con 0x7fd628102d30 2024-09-16T00:43:34.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.358+0000 7fd6257fa640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6140a32c0 0x7fd6140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:34.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.358+0000 7fd61ffff640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6140a32c0 0x7fd6140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:34.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.358+0000 7fd6257fa640 1 -- 172.21.15.31:0/457032502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd6180d03e0 con 0x7fd628102d30 2024-09-16T00:43:34.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.358+0000 7fd61ffff640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6140a32c0 0x7fd6140a5780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd61002a940 tx=0x7fd610005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:34.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.361+0000 7fd6257fa640 1 -- 172.21.15.31:0/457032502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd61809c6a0 con 0x7fd628102d30 2024-09-16T00:43:34.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.589+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd62810a2b0 con 0x7fd628102d30 2024-09-16T00:43:34.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.590+0000 7fd6257fa640 1 -- 172.21.15.31:0/457032502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fd6180a09a0 con 0x7fd628102d30 2024-09-16T00:43:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:43:34.591 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:43:34.591 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:43:34.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.592+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6140a32c0 msgr2=0x7fd6140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:34.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.592+0000 7fd62df9a640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6140a32c0 0x7fd6140a5780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd61002a940 tx=0x7fd610005a40 comp rx=0 tx=0).stop 2024-09-16T00:43:34.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.592+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd628102d30 msgr2=0x7fd628100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:34.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.592+0000 7fd62df9a640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd628102d30 0x7fd628100990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9f0 tx=0x7fd61800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.592+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 shutdown_connections 2024-09-16T00:43:34.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.592+0000 7fd62df9a640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6281082a0 0x7fd628101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.592+0000 7fd62df9a640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6140a32c0 0x7fd6140a5780 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.592+0000 7fd62df9a640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6281036e0 0x7fd628100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.592+0000 7fd62df9a640 1 --2- 172.21.15.31:0/457032502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd628102d30 0x7fd628100990 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:34.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.593+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 >> 172.21.15.31:0/457032502 conn(0x7fd6280fc7d0 msgr2=0x7fd628105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:34.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.593+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 shutdown_connections 2024-09-16T00:43:34.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:34.593+0000 7fd62df9a640 1 -- 172.21.15.31:0/457032502 wait complete. 2024-09-16T00:43:35.373 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 33404980 --release --await' 2024-09-16T00:43:35.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 33404980 --release --await 2024-09-16T00:43:35.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:35 smithi031 ceph-mon[30956]: pgmap v429: 129 pgs: 129 active+clean; 211 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 346 op/s 2024-09-16T00:43:35.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:35 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:35.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:35 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/457032502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:35 smithi081 ceph-mon[35539]: pgmap v429: 129 pgs: 129 active+clean; 211 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 346 op/s 2024-09-16T00:43:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:35 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:35 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/457032502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:35 smithi102 ceph-mon[35364]: pgmap v429: 129 pgs: 129 active+clean; 211 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 346 op/s 2024-09-16T00:43:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:35 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:43:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:35 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/457032502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:36.186 INFO:tasks.workunit.client.0.smithi031.stdout: 47 4753 0 3061 0 3693 0 2024-09-16T00:43:36.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.199+0000 7faff4f37640 1 -- 172.21.15.31:0/2405937281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0102dd0 msgr2=0x7faff01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.199+0000 7faff4f37640 1 --2- 172.21.15.31:0/2405937281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0102dd0 0x7faff01031b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009930 tx=0x7fafe002a420 comp rx=0 tx=0).stop 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.199+0000 7faff4f37640 1 -- 172.21.15.31:0/2405937281 shutdown_connections 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.199+0000 7faff4f37640 1 --2- 172.21.15.31:0/2405937281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faff0108300 0x7faff010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.199+0000 7faff4f37640 1 --2- 172.21.15.31:0/2405937281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff0103780 0x7faff0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.199+0000 7faff4f37640 1 --2- 172.21.15.31:0/2405937281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0102dd0 0x7faff01031b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.199+0000 7faff4f37640 1 -- 172.21.15.31:0/2405937281 >> 172.21.15.31:0/2405937281 conn(0x7faff00fc7d0 msgr2=0x7faff00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.199+0000 7faff4f37640 1 -- 172.21.15.31:0/2405937281 shutdown_connections 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.199+0000 7faff4f37640 1 -- 172.21.15.31:0/2405937281 wait complete. 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7faff4f37640 1 Processor -- start 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7faff4f37640 1 -- start start 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7faff4f37640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faff0102dd0 0x7faff0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7faff4f37640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0103780 0x7faff0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7faff4f37640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff0108300 0x7faff019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7faff4f37640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faff0112090 con 0x7faff0103780 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7faff4f37640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faff0111f10 con 0x7faff0108300 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7faff4f37640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faff0112210 con 0x7faff0102dd0 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0103780 0x7faff0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0103780 0x7faff0198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38948/0 (socket says 172.21.15.31:38948) 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7fafe7fff640 1 -- 172.21.15.31:0/2317969486 learned_addr learned my addr 172.21.15.31:0/2317969486 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.200+0000 7fafeed9d640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff0108300 0x7faff019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafee59c640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faff0102dd0 0x7faff0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafe7fff640 1 -- 172.21.15.31:0/2317969486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faff0102dd0 msgr2=0x7faff0198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafe7fff640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faff0102dd0 0x7faff0198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafe7fff640 1 -- 172.21.15.31:0/2317969486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff0108300 msgr2=0x7faff019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafe7fff640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff0108300 0x7faff019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafe7fff640 1 -- 172.21.15.31:0/2317969486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff019d600 con 0x7faff0103780 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafe7fff640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0103780 0x7faff0198b90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fafd800e9f0 tx=0x7fafd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafe77fe640 1 -- 172.21.15.31:0/2317969486 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd800ccc0 con 0x7faff0103780 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafe77fe640 1 -- 172.21.15.31:0/2317969486 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd8010910 con 0x7faff0103780 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7faff4f37640 1 -- 172.21.15.31:0/2317969486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff019d8f0 con 0x7faff0103780 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7fafe77fe640 1 -- 172.21.15.31:0/2317969486 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8018920 con 0x7faff0103780 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.201+0000 7faff4f37640 1 -- 172.21.15.31:0/2317969486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff01a51d0 con 0x7faff0103780 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.203+0000 7fafe77fe640 1 -- 172.21.15.31:0/2317969486 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fafd8010ac0 con 0x7faff0103780 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7faff7310740 1 Processor -- start 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7faff7310740 1 -- start start 2024-09-16T00:43:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7faff7310740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d42440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7faff7310740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7faff7310740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d4f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7faff7310740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c785ca7620 con 0x55c785d42060 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7faff7310740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c785c4ee50 con 0x55c785d42980 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7faff7310740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c785ca35e0 con 0x55c785d4d240 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7fafeed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d42440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7fafee59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7fafee59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:48482/0 (socket says 172.21.15.31:48482) 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7fafeed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d42440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38964/0 (socket says 172.21.15.31:38964) 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7fafee59c640 1 -- 172.21.15.31:0/2772828230 learned_addr learned my addr 172.21.15.31:0/2772828230 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7fafe77fe640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fafdc0a32c0 0x7fafdc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.204+0000 7fafe7fff640 1 --2- 172.21.15.31:0/2772828230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d4f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafeed9d640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fafdc0a32c0 0x7fafdc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafe77fe640 1 -- 172.21.15.31:0/2317969486 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fafd80d0000 con 0x7faff0103780 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafee59c640 1 -- 172.21.15.31:0/2772828230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 msgr2=0x55c785d4f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafee59c640 1 --2- 172.21.15.31:0/2772828230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d4f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafee59c640 1 -- 172.21.15.31:0/2772828230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 msgr2=0x55c785d42440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafee59c640 1 --2- 172.21.15.31:0/2772828230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d42440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafee59c640 1 -- 172.21.15.31:0/2772828230 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c785c3fd40 con 0x55c785d42980 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafeed9d640 1 --2- 172.21.15.31:0/2772828230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d42440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafee59c640 1 --2- 172.21.15.31:0/2772828230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4cd00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fafe002a940 tx=0x7fafe0005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c271bb4867dbafb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafeed9d640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fafdc0a32c0 0x7fafdc0a5780 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fafe8005fd0 tx=0x7fafe8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafe4ff9640 1 -- 172.21.15.31:0/2772828230 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe003f070 con 0x55c785d42980 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafe4ff9640 1 -- 172.21.15.31:0/2772828230 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe0004030 con 0x55c785d42980 2024-09-16T00:43:36.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.205+0000 7fafe4ff9640 1 -- 172.21.15.31:0/2772828230 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0035070 con 0x55c785d42980 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 -- 172.21.15.31:0/2772828230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 msgr2=0x55c785d4cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 --2- 172.21.15.31:0/2772828230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4cd00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fafe002a940 tx=0x7fafe0005dd0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 -- 172.21.15.31:0/2772828230 shutdown_connections 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 --2- 172.21.15.31:0/2772828230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d4f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 --2- 172.21.15.31:0/2772828230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4cd00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 --2- 172.21.15.31:0/2772828230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d42440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 -- 172.21.15.31:0/2772828230 >> 172.21.15.31:0/2772828230 conn(0x55c785d3e5f0 msgr2=0x55c785d3e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 -- 172.21.15.31:0/2772828230 shutdown_connections 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 -- 172.21.15.31:0/2772828230 wait complete. 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 -- start start 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.206+0000 7faff7310740 1 Processor -- start 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7faff7310740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d4a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7faff7310740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7faff7310740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d4ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7faff7310740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c785d4fe10 con 0x55c785d42060 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7faff7310740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c785c2ff10 con 0x55c785d42980 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7faff7310740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c785d4ff90 con 0x55c785d4d240 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7faff7310740 10 client.0 _ll_register_callbacks cb 0x55c785c53db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d4a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d4a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38972/0 (socket says 172.21.15.31:38972) 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafee59c640 1 -- 172.21.15.31:0/3409994844 learned_addr learned my addr 172.21.15.31:0/3409994844 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:48496/0 (socket says 172.21.15.31:48496) 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafeed9d640 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d4ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafee59c640 1 -- 172.21.15.31:0/3409994844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 msgr2=0x55c785d4ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafee59c640 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d4ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafee59c640 1 -- 172.21.15.31:0/3409994844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 msgr2=0x55c785d4a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafee59c640 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafee59c640 1 -- 172.21.15.31:0/3409994844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7859b6580 con 0x55c785d42060 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafee59c640 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d4a0c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fafe002aaf0 tx=0x7fafe0032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7128672fe27d2031 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.207+0000 7fafd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3409994844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0035070 con 0x55c785d42060 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3409994844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fafe00317b0 con 0x55c785d42060 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7faff7310740 1 -- 172.21.15.31:0/3409994844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe0004030 con 0x55c785d42060 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3409994844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fafe0055050 con 0x55c785d42060 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7faff7310740 10 client.15657 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:36.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7faff7310740 1 -- 172.21.15.31:0/3409994844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c785a80960 con 0x55c785d42060 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3409994844 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe0004030 con 0x55c785d42060 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3409994844 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fafe00047e0 con 0x55c785d42060 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7faff7310740 10 client.15657 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7faff7310740 10 client.15657 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7faff7310740 20 client.15657 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7faff7310740 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c785d5cbd0 0x55c785d5efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7faff7310740 4 client.15657 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7faff7310740 1 -- 172.21.15.31:0/3409994844 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c785cb4b90 con 0x55c785d5cbd0 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.208+0000 7fafe7fff640 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c785d5cbd0 0x55c785d5efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.209+0000 7fafd5ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.209+0000 7fafe7fff640 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c785d5cbd0 0x55c785d5efc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0c256b32abf3b8 server_cookie=24b478aa75eac9bc in_seq=0 out_seq=0 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3409994844 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c785cb4b90 con 0x55c785d5cbd0 2024-09-16T00:43:36.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7fafd5ffb640 10 client.15657 handle_command_reply: tid=0 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 client.15657 shutdown 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 -- 172.21.15.31:0/3409994844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 msgr2=0x55c785d4a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d4a0c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fafe002aaf0 tx=0x7fafe0032f60 comp rx=0 tx=0).stop 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 -- 172.21.15.31:0/3409994844 shutdown_connections 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c785d5cbd0 0x55c785d5efc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d4ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7fafd5ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 --2- 172.21.15.31:0/3409994844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42060 0x55c785d4a0c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 -- 172.21.15.31:0/3409994844 >> 172.21.15.31:0/3409994844 conn(0x55c785d3e5f0 msgr2=0x55c785d3d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.210+0000 7faff7310740 1 -- 172.21.15.31:0/3409994844 shutdown_connections 2024-09-16T00:43:36.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.211+0000 7faff7310740 1 -- 172.21.15.31:0/3409994844 wait complete. 2024-09-16T00:43:36.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.211+0000 7faff7310740 20 client.15657 trim_cache size 0 max 16384 2024-09-16T00:43:36.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.228+0000 7faff7310740 1 Processor -- start 2024-09-16T00:43:36.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.228+0000 7faff7310740 1 -- start start 2024-09-16T00:43:36.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.228+0000 7faff7310740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d42060 0x55c785d3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.228+0000 7faff7310740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7faff7310740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d4d240 0x55c785d4acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7faff7310740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faff01aab10 con 0x55c785d4d240 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7faff7310740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faff01ad390 con 0x55c785d42980 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7faff7310740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faff01ab180 con 0x55c785d42060 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafeed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d4d240 0x55c785d4acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafeed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d4d240 0x55c785d4acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38986/0 (socket says 172.21.15.31:38986) 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafeed9d640 1 -- 172.21.15.31:0/1062542305 learned_addr learned my addr 172.21.15.31:0/1062542305 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafee59c640 1 --2- 172.21.15.31:0/1062542305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d42060 0x55c785d3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafeed9d640 1 -- 172.21.15.31:0/1062542305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d42060 msgr2=0x55c785d3da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafeed9d640 1 --2- 172.21.15.31:0/1062542305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d42060 0x55c785d3da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafeed9d640 1 -- 172.21.15.31:0/1062542305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 msgr2=0x55c785d4a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafeed9d640 1 --2- 172.21.15.31:0/1062542305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafeed9d640 1 -- 172.21.15.31:0/1062542305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c785d3e420 con 0x55c785d4d240 2024-09-16T00:43:36.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafee59c640 1 --2- 172.21.15.31:0/1062542305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d42060 0x55c785d3da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:36.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafeed9d640 1 --2- 172.21.15.31:0/1062542305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d4d240 0x55c785d4acd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fafe8012a00 tx=0x7fafe8009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b33bc69f65f6441 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafd7fff640 1 -- 172.21.15.31:0/1062542305 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe801a070 con 0x55c785d4d240 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.229+0000 7fafd7fff640 1 -- 172.21.15.31:0/1062542305 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe8026030 con 0x55c785d4d240 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7fafd7fff640 1 -- 172.21.15.31:0/1062542305 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8022790 con 0x55c785d4d240 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7faff7310740 1 -- 172.21.15.31:0/1062542305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d4d240 msgr2=0x55c785d4acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7faff7310740 1 --2- 172.21.15.31:0/1062542305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d4d240 0x55c785d4acd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fafe8012a00 tx=0x7fafe8009ab0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7faff7310740 1 -- 172.21.15.31:0/1062542305 shutdown_connections 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7faff7310740 1 --2- 172.21.15.31:0/1062542305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d4d240 0x55c785d4acd0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7faff7310740 1 --2- 172.21.15.31:0/1062542305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42980 0x55c785d4a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7faff7310740 1 --2- 172.21.15.31:0/1062542305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d42060 0x55c785d3da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7faff7310740 1 -- 172.21.15.31:0/1062542305 >> 172.21.15.31:0/1062542305 conn(0x55c785d3e5f0 msgr2=0x55c785d4a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7faff7310740 1 -- 172.21.15.31:0/1062542305 shutdown_connections 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.230+0000 7faff7310740 1 -- 172.21.15.31:0/1062542305 wait complete. 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7faff7310740 1 -- start start 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7faff7310740 1 Processor -- start 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7faff7310740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42060 0x55c785d551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7faff7310740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42980 0x55c785d55720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7faff7310740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7faff7310740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faff01ab850 con 0x55c785d42980 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7faff7310740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faff01accc0 con 0x55c785d42060 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7faff7310740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faff01abf20 con 0x55c785d4d240 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7faff7310740 10 client.0 _ll_register_callbacks cb 0x7fafe0005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7fafee59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42060 0x55c785d551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7fafee59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42060 0x55c785d551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:48516/0 (socket says 172.21.15.31:48516) 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7fafee59c640 1 -- 172.21.15.31:0/3312352731 learned_addr learned my addr 172.21.15.31:0/3312352731 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42980 0x55c785d55720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.231+0000 7fafeed9d640 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafee59c640 1 -- 172.21.15.31:0/3312352731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 msgr2=0x55c785d521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafee59c640 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafee59c640 1 -- 172.21.15.31:0/3312352731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42980 msgr2=0x55c785d55720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafee59c640 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42980 0x55c785d55720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafee59c640 1 -- 172.21.15.31:0/3312352731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c785d44ed0 con 0x55c785d42060 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafeed9d640 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafee59c640 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42060 0x55c785d551e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fafe0004da0 tx=0x7fafe00051c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85b54b0aebd0a6e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3312352731 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe005e060 con 0x55c785d42060 2024-09-16T00:43:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3312352731 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fafe00315b0 con 0x55c785d42060 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7faff7310740 1 -- 172.21.15.31:0/3312352731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe8026030 con 0x55c785d42060 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3312352731 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fafe0065050 con 0x55c785d42060 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7faff7310740 10 client.25349 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.232+0000 7faff7310740 1 -- 172.21.15.31:0/3312352731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c785dc7a70 con 0x55c785d42060 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3312352731 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe8026030 con 0x55c785d42060 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3312352731 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fafe003b5c0 con 0x55c785d42060 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7faff7310740 10 client.25349 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7faff7310740 10 client.25349 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7faff7310740 20 client.25349 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7faff7310740 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c785dc8160 0x55c785dca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7faff7310740 4 client.25349 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "33404980", "release": true, "await": true}] 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7faff7310740 1 -- 172.21.15.31:0/3312352731 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "set_id": "33404980", "release": true, "await": true}) -- 0x55c785caf0a0 con 0x55c785dc8160 2024-09-16T00:43:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7fafe7fff640 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c785dc8160 0x55c785dca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7fafe7fff640 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c785dc8160 0x55c785dca580 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93516627bcd06238 server_cookie=4c6209733f2de930 in_seq=0 out_seq=0 2024-09-16T00:43:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.233+0000 7fafd5ffb640 10 client.25349 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.722+0000 7fafd5ffb640 1 -- 172.21.15.31:0/3312352731 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c785caf0a0 con 0x55c785dc8160 2024-09-16T00:43:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.722+0000 7fafd5ffb640 10 client.25349 handle_command_reply: tid=0 2024-09-16T00:43:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.722+0000 7faff7310740 1 client.25349 shutdown 2024-09-16T00:43:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 -- 172.21.15.31:0/3312352731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42060 msgr2=0x55c785d551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42060 0x55c785d551e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fafe0004da0 tx=0x7fafe00051c0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 -- 172.21.15.31:0/3312352731 shutdown_connections 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55c785dc8160 0x55c785dca580 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c785d4d240 0x55c785d521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7fafd5ffb640 0 client.25349 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c785d42980 0x55c785d55720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 --2- 172.21.15.31:0/3312352731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c785d42060 0x55c785d551e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 -- 172.21.15.31:0/3312352731 >> 172.21.15.31:0/3312352731 conn(0x55c785d3e5f0 msgr2=0x55c785d4c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 -- 172.21.15.31:0/3312352731 shutdown_connections 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 1 -- 172.21.15.31:0/3312352731 wait complete. 2024-09-16T00:43:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.723+0000 7faff7310740 20 client.25349 trim_cache size 0 max 16384 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 8, 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "33404980": { 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 8, 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:43:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 139.6, 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:36.726 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:43:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 -- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fafdc0a32c0 msgr2=0x7fafdc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fafdc0a32c0 0x7fafdc0a5780 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fafe8005fd0 tx=0x7fafe8004ab0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 -- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0103780 msgr2=0x7faff0198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0103780 0x7faff0198b90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fafd800e9f0 tx=0x7fafd800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 -- 172.21.15.31:0/2317969486 shutdown_connections 2024-09-16T00:43:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff0108300 0x7faff019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faff0103780 0x7faff0198b90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fafdc0a32c0 0x7fafdc0a5780 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 --2- 172.21.15.31:0/2317969486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faff0102dd0 0x7faff0198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 -- 172.21.15.31:0/2317969486 >> 172.21.15.31:0/2317969486 conn(0x7faff00fc7d0 msgr2=0x7faff0105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 -- 172.21.15.31:0/2317969486 shutdown_connections 2024-09-16T00:43:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:36.724+0000 7faff4f37640 1 -- 172.21.15.31:0/2317969486 wait complete. 2024-09-16T00:43:37.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:37 smithi031 ceph-mon[30956]: pgmap v430: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-09-16T00:43:37.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:37 smithi081 ceph-mon[35539]: pgmap v430: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-09-16T00:43:37.791 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '33404980', total seconds elapsed: 23.8 2024-09-16T00:43:37.791 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-16T00:43:37.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:43:37.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:37 smithi102 ceph-mon[35364]: pgmap v430: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-09-16T00:43:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:38.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:38 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:43:38.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.593+0000 7fcb8348f640 1 -- 172.21.15.31:0/3063951638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c106a80 msgr2=0x7fcb7c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.593+0000 7fcb8348f640 1 --2- 172.21.15.31:0/3063951638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c106a80 0x7fcb7c106e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009930 tx=0x7fcb7802a420 comp rx=0 tx=0).stop 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.593+0000 7fcb8348f640 1 -- 172.21.15.31:0/3063951638 shutdown_connections 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.593+0000 7fcb8348f640 1 --2- 172.21.15.31:0/3063951638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb7c101420 0x7fcb7c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.593+0000 7fcb8348f640 1 --2- 172.21.15.31:0/3063951638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb7c100a80 0x7fcb7c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.593+0000 7fcb8348f640 1 --2- 172.21.15.31:0/3063951638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c106a80 0x7fcb7c106e60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.593+0000 7fcb8348f640 1 -- 172.21.15.31:0/3063951638 >> 172.21.15.31:0/3063951638 conn(0x7fcb7c0fc7a0 msgr2=0x7fcb7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.593+0000 7fcb8348f640 1 -- 172.21.15.31:0/3063951638 shutdown_connections 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.593+0000 7fcb8348f640 1 -- 172.21.15.31:0/3063951638 wait complete. 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.594+0000 7fcb8348f640 1 Processor -- start 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.594+0000 7fcb8348f640 1 -- start start 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.594+0000 7fcb8348f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c100a80 0x7fcb7c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:38.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.594+0000 7fcb8348f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb7c101420 0x7fcb7c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.594+0000 7fcb8348f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb7c106a80 0x7fcb7c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.594+0000 7fcb8348f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c110e80 con 0x7fcb7c100a80 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8348f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c110d00 con 0x7fcb7c106a80 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8348f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c111000 con 0x7fcb7c101420 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8122b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c100a80 0x7fcb7c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8122b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c100a80 0x7fcb7c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48732/0 (socket says 172.21.15.31:48732) 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8122b640 1 -- 172.21.15.31:0/2908066203 learned_addr learned my addr 172.21.15.31:0/2908066203 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb81a2c640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb7c106a80 0x7fcb7c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb80a2a640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb7c101420 0x7fcb7c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8122b640 1 -- 172.21.15.31:0/2908066203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb7c101420 msgr2=0x7fcb7c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8122b640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb7c101420 0x7fcb7c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8122b640 1 -- 172.21.15.31:0/2908066203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb7c106a80 msgr2=0x7fcb7c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8122b640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb7c106a80 0x7fcb7c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8122b640 1 -- 172.21.15.31:0/2908066203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c19d360 con 0x7fcb7c100a80 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8122b640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c100a80 0x7fcb7c1983b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009230 tx=0x7fcb78004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb727fc640 1 -- 172.21.15.31:0/2908066203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7803f030 con 0x7fcb7c100a80 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c19d5f0 con 0x7fcb7c100a80 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb727fc640 1 -- 172.21.15.31:0/2908066203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb78003ad0 con 0x7fcb7c100a80 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb727fc640 1 -- 172.21.15.31:0/2908066203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78035070 con 0x7fcb7c100a80 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.595+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c1a4f30 con 0x7fcb7c100a80 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.596+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c100ee0 con 0x7fcb7c100a80 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.598+0000 7fcb727fc640 1 -- 172.21.15.31:0/2908066203 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcb78042030 con 0x7fcb7c100a80 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.598+0000 7fcb727fc640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb680a3240 0x7fcb680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.599+0000 7fcb80a2a640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb680a3240 0x7fcb680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.599+0000 7fcb727fc640 1 -- 172.21.15.31:0/2908066203 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fcb780f0240 con 0x7fcb7c100a80 2024-09-16T00:43:38.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.599+0000 7fcb80a2a640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb680a3240 0x7fcb680a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0fece0 tx=0x7fcb6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.602+0000 7fcb727fc640 1 -- 172.21.15.31:0/2908066203 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb780bc560 con 0x7fcb7c100a80 2024-09-16T00:43:38.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:38 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:43:38.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:38 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:43:38.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.831+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb7c199710 con 0x7fcb7c100a80 2024-09-16T00:43:38.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.832+0000 7fcb727fc640 1 -- 172.21.15.31:0/2908066203 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fcb7c199710 con 0x7fcb7c100a80 2024-09-16T00:43:38.833 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:43:38.833 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:43:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:43:38.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.835+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb680a3240 msgr2=0x7fcb680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:38.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.835+0000 7fcb8348f640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb680a3240 0x7fcb680a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0fece0 tx=0x7fcb6c005f90 comp rx=0 tx=0).stop 2024-09-16T00:43:38.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.835+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c100a80 msgr2=0x7fcb7c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:38.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.835+0000 7fcb8348f640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c100a80 0x7fcb7c1983b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009230 tx=0x7fcb78004270 comp rx=0 tx=0).stop 2024-09-16T00:43:38.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.836+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 shutdown_connections 2024-09-16T00:43:38.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.836+0000 7fcb8348f640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb7c106a80 0x7fcb7c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:38.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.836+0000 7fcb8348f640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb680a3240 0x7fcb680a5700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:38.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.836+0000 7fcb8348f640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb7c101420 0x7fcb7c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:38.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.836+0000 7fcb8348f640 1 --2- 172.21.15.31:0/2908066203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c100a80 0x7fcb7c1983b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:38.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.836+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 >> 172.21.15.31:0/2908066203 conn(0x7fcb7c0fc7a0 msgr2=0x7fcb7c06a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:38.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.836+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 shutdown_connections 2024-09-16T00:43:38.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:38.836+0000 7fcb8348f640 1 -- 172.21.15.31:0/2908066203 wait complete. 2024-09-16T00:43:39.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:39 smithi031 ceph-mon[30956]: pgmap v431: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 314 op/s 2024-09-16T00:43:39.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:39 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2908066203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:39.633 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-16T00:43:39.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-16T00:43:39.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:39 smithi081 ceph-mon[35539]: pgmap v431: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 314 op/s 2024-09-16T00:43:39.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:39 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2908066203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:39.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:39 smithi102 ceph-mon[35364]: pgmap v431: 129 pgs: 129 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 314 op/s 2024-09-16T00:43:39.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:39 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2908066203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.407+0000 7f434c44d640 1 -- 172.21.15.31:0/3724801274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102570 msgr2=0x7f43441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.407+0000 7f434c44d640 1 --2- 172.21.15.31:0/3724801274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102570 0x7f43441029b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4338009900 tx=0x7f433802a480 comp rx=0 tx=0).stop 2024-09-16T00:43:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.407+0000 7f434c44d640 1 -- 172.21.15.31:0/3724801274 shutdown_connections 2024-09-16T00:43:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.407+0000 7f434c44d640 1 --2- 172.21.15.31:0/3724801274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4344102ef0 0x7f434410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.407+0000 7f434c44d640 1 --2- 172.21.15.31:0/3724801274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102570 0x7f43441029b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.407+0000 7f434c44d640 1 --2- 172.21.15.31:0/3724801274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4344100e20 0x7f4344101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.407+0000 7f434c44d640 1 -- 172.21.15.31:0/3724801274 >> 172.21.15.31:0/3724801274 conn(0x7f43440fc7d0 msgr2=0x7f43440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.407+0000 7f434c44d640 1 -- 172.21.15.31:0/3724801274 shutdown_connections 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.407+0000 7f434c44d640 1 -- 172.21.15.31:0/3724801274 wait complete. 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434c44d640 1 Processor -- start 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434c44d640 1 -- start start 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434c44d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4344100e20 0x7f43440fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434c44d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4344102570 0x7f43440fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434c44d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102ef0 0x7f43440fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434c44d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4344110f80 con 0x7f4344102ef0 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434c44d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4344110e00 con 0x7f4344102570 2024-09-16T00:43:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434c44d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4344111100 con 0x7f4344100e20 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434a9ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102ef0 0x7f43440fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f43499e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4344102570 0x7f43440fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434a9ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102ef0 0x7f43440fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48748/0 (socket says 172.21.15.31:48748) 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434a9ea640 1 -- 172.21.15.31:0/2998253269 learned_addr learned my addr 172.21.15.31:0/2998253269 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.408+0000 7f434a1e9640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4344100e20 0x7f43440fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f434a1e9640 1 -- 172.21.15.31:0/2998253269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4344102570 msgr2=0x7f43440fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f434a1e9640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4344102570 0x7f43440fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f434a1e9640 1 -- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102ef0 msgr2=0x7f43440fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f434a1e9640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102ef0 0x7f43440fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f434a1e9640 1 -- 172.21.15.31:0/2998253269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43441a69a0 con 0x7f4344100e20 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f434a9ea640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102ef0 0x7f43440fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f434a1e9640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4344100e20 0x7f43440fcc80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f433400eb00 tx=0x7f433400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f43437fe640 1 -- 172.21.15.31:0/2998253269 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334010430 con 0x7f4344100e20 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f434c44d640 1 -- 172.21.15.31:0/2998253269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43441a6c90 con 0x7f4344100e20 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f43437fe640 1 -- 172.21.15.31:0/2998253269 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4334010ab0 con 0x7f4344100e20 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f43437fe640 1 -- 172.21.15.31:0/2998253269 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334017bb0 con 0x7f4344100e20 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.409+0000 7f434c44d640 1 -- 172.21.15.31:0/2998253269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441a71a0 con 0x7f4344100e20 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.412+0000 7f43437fe640 1 -- 172.21.15.31:0/2998253269 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4334010c60 con 0x7f4344100e20 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.412+0000 7f43437fe640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f432c0a3240 0x7f432c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434e826740 1 Processor -- start 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f43499e8640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f432c0a3240 0x7f432c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f43437fe640 1 -- 172.21.15.31:0/2998253269 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f43340d0670 con 0x7f4344100e20 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434e826740 1 -- start start 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434e826740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 0x56380ed6ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434e826740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6b170 0x56380ed754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434e826740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed75a30 0x56380ed77e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434e826740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56380eccf160 con 0x56380ed6b170 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434e826740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56380ec76720 con 0x56380ed75a30 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434e826740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56380eccb120 con 0x56380ed6a850 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f43499e8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6b170 0x56380ed754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434a1e9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 0x56380ed6ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434a9ea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed75a30 0x56380ed77e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434a1e9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 0x56380ed6ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:40512/0 (socket says 172.21.15.31:40512) 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.413+0000 7f434a1e9640 1 -- 172.21.15.31:0/2377144820 learned_addr learned my addr 172.21.15.31:0/2377144820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f434a9ea640 1 -- 172.21.15.31:0/2377144820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 msgr2=0x56380ed6ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f434a9ea640 1 --2- 172.21.15.31:0/2377144820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 0x56380ed6ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f434a9ea640 1 -- 172.21.15.31:0/2377144820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6b170 msgr2=0x56380ed754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f434a9ea640 1 --2- 172.21.15.31:0/2377144820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6b170 0x56380ed754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f434a9ea640 1 -- 172.21.15.31:0/2377144820 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380ea40ba0 con 0x56380ed75a30 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f43499e8640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f432c0a3240 0x7f432c0a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4338009230 tx=0x7f4338004370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f434a9ea640 1 --2- 172.21.15.31:0/2377144820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed75a30 0x56380ed77e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f433c011440 tx=0x7f433c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f9765b27acf1556 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f4340dea640 1 -- 172.21.15.31:0/2377144820 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c01e070 con 0x56380ed75a30 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f4340dea640 1 -- 172.21.15.31:0/2377144820 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c00fa20 con 0x56380ed75a30 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.414+0000 7f4340dea640 1 -- 172.21.15.31:0/2377144820 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c019b00 con 0x56380ed75a30 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 -- 172.21.15.31:0/2377144820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed75a30 msgr2=0x56380ed77e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 --2- 172.21.15.31:0/2377144820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed75a30 0x56380ed77e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f433c011440 tx=0x7f433c011900 comp rx=0 tx=0).stop 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 -- 172.21.15.31:0/2377144820 shutdown_connections 2024-09-16T00:43:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 --2- 172.21.15.31:0/2377144820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed75a30 0x56380ed77e20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 --2- 172.21.15.31:0/2377144820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6b170 0x56380ed754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 --2- 172.21.15.31:0/2377144820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 0x56380ed6ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 -- 172.21.15.31:0/2377144820 >> 172.21.15.31:0/2377144820 conn(0x56380ed66de0 msgr2=0x56380ed671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 -- 172.21.15.31:0/2377144820 shutdown_connections 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 -- 172.21.15.31:0/2377144820 wait complete. 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 -- start start 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.415+0000 7f434e826740 1 Processor -- start 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434e826740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6a850 0x56380ed728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434e826740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6b170 0x56380ed72e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434e826740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380ed73360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434e826740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56380ed78600 con 0x56380ed75a30 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434e826740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56380ec57900 con 0x56380ed6a850 2024-09-16T00:43:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434e826740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56380ed78780 con 0x56380ed6b170 2024-09-16T00:43:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434e826740 10 client.0 _ll_register_callbacks cb 0x56380ec55780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a9ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380ed73360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a1e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6a850 0x56380ed728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a1e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6a850 0x56380ed728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:47592/0 (socket says 172.21.15.31:47592) 2024-09-16T00:43:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a1e9640 1 -- 172.21.15.31:0/2237419738 learned_addr learned my addr 172.21.15.31:0/2237419738 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f43499e8640 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6b170 0x56380ed72e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a9ea640 1 -- 172.21.15.31:0/2237419738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6b170 msgr2=0x56380ed72e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a9ea640 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6b170 0x56380ed72e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a9ea640 1 -- 172.21.15.31:0/2237419738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6a850 msgr2=0x56380ed728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a9ea640 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6a850 0x56380ed728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a9ea640 1 -- 172.21.15.31:0/2237419738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380ec97c50 con 0x56380ed75a30 2024-09-16T00:43:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.416+0000 7f434a1e9640 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6a850 0x56380ed728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f434a9ea640 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380ed73360 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f433c011e00 tx=0x7f433c005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e48e4fa0e40cdd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f4329ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f4329ffb640 1 -- 172.21.15.31:0/2237419738 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c01e040 con 0x56380ed75a30 2024-09-16T00:43:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f4329ffb640 1 -- 172.21.15.31:0/2237419738 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f433c005050 con 0x56380ed75a30 2024-09-16T00:43:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f434e826740 1 -- 172.21.15.31:0/2237419738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433c00fa20 con 0x56380ed75a30 2024-09-16T00:43:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f4329ffb640 1 -- 172.21.15.31:0/2237419738 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f433c030050 con 0x56380ed75a30 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f434e826740 10 client.15681 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f434e826740 1 -- 172.21.15.31:0/2237419738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56380ed84280 con 0x56380ed75a30 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f4329ffb640 1 -- 172.21.15.31:0/2237419738 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c00fa20 con 0x56380ed75a30 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f4329ffb640 1 -- 172.21.15.31:0/2237419738 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f433c0046c0 con 0x56380ed75a30 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f434e826740 10 client.15681 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.417+0000 7f434e826740 10 client.15681 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.418+0000 7f434e826740 20 client.15681 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.418+0000 7f434e826740 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56380ed856e0 0x56380ed87ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.418+0000 7f434e826740 4 client.15681 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.418+0000 7f434e826740 1 -- 172.21.15.31:0/2237419738 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56380ecdc890 con 0x56380ed856e0 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.418+0000 7f434a1e9640 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56380ed856e0 0x56380ed87ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.418+0000 7f4329ffb640 10 client.15681 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:40.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.418+0000 7f434a1e9640 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56380ed856e0 0x56380ed87ad0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676c6e6861d8d464 server_cookie=b9cec28e8355f479 in_seq=0 out_seq=0 2024-09-16T00:43:40.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f4329ffb640 1 -- 172.21.15.31:0/2237419738 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56380ecdc890 con 0x56380ed856e0 2024-09-16T00:43:40.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f4329ffb640 10 client.15681 handle_command_reply: tid=0 2024-09-16T00:43:40.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 client.15681 shutdown 2024-09-16T00:43:40.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 -- 172.21.15.31:0/2237419738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 msgr2=0x56380ed73360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:40.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380ed73360 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f433c011e00 tx=0x7f433c005cc0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 -- 172.21.15.31:0/2237419738 shutdown_connections 2024-09-16T00:43:40.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380ed73360 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56380ed856e0 0x56380ed87ad0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6b170 0x56380ed72e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 --2- 172.21.15.31:0/2237419738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6a850 0x56380ed728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 -- 172.21.15.31:0/2237419738 >> 172.21.15.31:0/2237419738 conn(0x56380ed66de0 msgr2=0x56380ed6cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f4329ffb640 0 client.15681 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 -- 172.21.15.31:0/2237419738 shutdown_connections 2024-09-16T00:43:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 1 -- 172.21.15.31:0/2237419738 wait complete. 2024-09-16T00:43:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.420+0000 7f434e826740 20 client.15681 trim_cache size 0 max 16384 2024-09-16T00:43:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434e826740 1 Processor -- start 2024-09-16T00:43:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434e826740 1 -- start start 2024-09-16T00:43:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434e826740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6a850 0x56380ed718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434e826740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 0x56380ed8d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434e826740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed75a30 0x56380ed8d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434e826740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43441aef10 con 0x56380ed6a850 2024-09-16T00:43:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434e826740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f43441b1790 con 0x56380ed6b170 2024-09-16T00:43:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434e826740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f43441af580 con 0x56380ed75a30 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434a1e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6a850 0x56380ed718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434a1e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6a850 0x56380ed718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48774/0 (socket says 172.21.15.31:48774) 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434a1e9640 1 -- 172.21.15.31:0/3233694165 learned_addr learned my addr 172.21.15.31:0/3233694165 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f43499e8640 1 --2- 172.21.15.31:0/3233694165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 0x56380ed8d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.439+0000 7f434a9ea640 1 --2- 172.21.15.31:0/3233694165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed75a30 0x56380ed8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.440+0000 7f434a1e9640 1 -- 172.21.15.31:0/3233694165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed75a30 msgr2=0x56380ed8d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.440+0000 7f434a1e9640 1 --2- 172.21.15.31:0/3233694165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed75a30 0x56380ed8d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.440+0000 7f434a1e9640 1 -- 172.21.15.31:0/3233694165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 msgr2=0x56380ed8d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.440+0000 7f434a1e9640 1 --2- 172.21.15.31:0/3233694165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 0x56380ed8d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.440+0000 7f434a1e9640 1 -- 172.21.15.31:0/3233694165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380ed70e30 con 0x56380ed6a850 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.440+0000 7f434a1e9640 1 --2- 172.21.15.31:0/3233694165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6a850 0x56380ed718c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f433409c5e0 tx=0x7f43340cdc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e1362f51d7c694 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.440+0000 7f432bfff640 1 -- 172.21.15.31:0/3233694165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43340bb070 con 0x56380ed6a850 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.440+0000 7f432bfff640 1 -- 172.21.15.31:0/3233694165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43340a0c90 con 0x56380ed6a850 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.440+0000 7f432bfff640 1 -- 172.21.15.31:0/3233694165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433409ac70 con 0x56380ed6a850 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 -- 172.21.15.31:0/3233694165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6a850 msgr2=0x56380ed718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 --2- 172.21.15.31:0/3233694165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6a850 0x56380ed718c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f433409c5e0 tx=0x7f43340cdc90 comp rx=0 tx=0).stop 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 -- 172.21.15.31:0/3233694165 shutdown_connections 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 --2- 172.21.15.31:0/3233694165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed75a30 0x56380ed8d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 --2- 172.21.15.31:0/3233694165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 0x56380ed8d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 --2- 172.21.15.31:0/3233694165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed6a850 0x56380ed718c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 -- 172.21.15.31:0/3233694165 >> 172.21.15.31:0/3233694165 conn(0x56380ed66de0 msgr2=0x56380ed711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 -- 172.21.15.31:0/3233694165 shutdown_connections 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 -- 172.21.15.31:0/3233694165 wait complete. 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 -- start start 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.441+0000 7f434e826740 1 Processor -- start 2024-09-16T00:43:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434e826740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 0x56380ed78a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434e826740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 0x56380ed78fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434e826740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380edee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434e826740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43441afc50 con 0x56380ed75a30 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434e826740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f43441b10c0 con 0x56380ed6b170 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434e826740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f43441b0320 con 0x56380ed6a850 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434e826740 10 client.0 _ll_register_callbacks cb 0x7f433c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a9ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380edee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f43499e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 0x56380ed78fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a9ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380edee1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48780/0 (socket says 172.21.15.31:48780) 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a9ea640 1 -- 172.21.15.31:0/4013584063 learned_addr learned my addr 172.21.15.31:0/4013584063 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a1e9640 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 0x56380ed78a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a9ea640 1 -- 172.21.15.31:0/4013584063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 msgr2=0x56380ed78a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:40.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a9ea640 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 0x56380ed78a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a9ea640 1 -- 172.21.15.31:0/4013584063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 msgr2=0x56380ed78fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a9ea640 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 0x56380ed78fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a9ea640 1 -- 172.21.15.31:0/4013584063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380edee710 con 0x56380ed75a30 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f43499e8640 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 0x56380ed78fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f434a9ea640 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380edee1d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f433c011e00 tx=0x7f433c004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbe4322815dd36c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.442+0000 7f4329ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f4329ffb640 1 -- 172.21.15.31:0/4013584063 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c01e040 con 0x56380ed75a30 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f4329ffb640 1 -- 172.21.15.31:0/4013584063 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f433c0195d0 con 0x56380ed75a30 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f434e826740 1 -- 172.21.15.31:0/4013584063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43340a0c90 con 0x56380ed75a30 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f4329ffb640 1 -- 172.21.15.31:0/4013584063 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f433c019810 con 0x56380ed75a30 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f434e826740 10 client.15687 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f434e826740 1 -- 172.21.15.31:0/4013584063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56380edef870 con 0x56380ed75a30 2024-09-16T00:43:40.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f4329ffb640 1 -- 172.21.15.31:0/4013584063 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43340a0c90 con 0x56380ed75a30 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f4329ffb640 1 -- 172.21.15.31:0/4013584063 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f433c004f10 con 0x56380ed75a30 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f434e826740 10 client.15687 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f434e826740 10 client.15687 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f434e826740 20 client.15687 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f434e826740 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56380edeff90 0x56380edf2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f434e826740 4 client.15687 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.443+0000 7f434e826740 1 -- 172.21.15.31:0/4013584063 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x56380ecd6be0 con 0x56380edeff90 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.444+0000 7f434a1e9640 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56380edeff90 0x56380edf2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.444+0000 7f434a1e9640 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56380edeff90 0x56380edf2380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc5c9322d2103f85 server_cookie=fe1c8e98913d5a05 in_seq=0 out_seq=0 2024-09-16T00:43:40.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:40.444+0000 7f4329ffb640 10 client.15687 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:41.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:41 smithi031 ceph-mon[30956]: pgmap v432: 129 pgs: 129 active+clean; 217 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-16T00:43:41.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:41 smithi081 ceph-mon[35539]: pgmap v432: 129 pgs: 129 active+clean; 217 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-16T00:43:41.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:41 smithi102 ceph-mon[35364]: pgmap v432: 129 pgs: 129 active+clean; 217 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-16T00:43:43.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:43 smithi031 ceph-mon[30956]: pgmap v433: 129 pgs: 129 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-09-16T00:43:43.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:43 smithi081 ceph-mon[35539]: pgmap v433: 129 pgs: 129 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-09-16T00:43:43.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:43 smithi102 ceph-mon[35364]: pgmap v433: 129 pgs: 129 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-09-16T00:43:45.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:45 smithi031 ceph-mon[30956]: pgmap v434: 129 pgs: 129 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-16T00:43:45.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:45 smithi081 ceph-mon[35539]: pgmap v434: 129 pgs: 129 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-16T00:43:45.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:45 smithi102 ceph-mon[35364]: pgmap v434: 129 pgs: 129 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-16T00:43:46.187 INFO:tasks.workunit.client.0.smithi031.stdout: 47 13303 16 7736 97 5325 136 2024-09-16T00:43:47.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.260+0000 7f4329ffb640 1 -- 172.21.15.31:0/4013584063 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56380ecd6be0 con 0x56380edeff90 2024-09-16T00:43:47.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.260+0000 7f4329ffb640 10 client.15687 handle_command_reply: tid=0 2024-09-16T00:43:47.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.260+0000 7f434e826740 1 client.15687 shutdown 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 -- 172.21.15.31:0/4013584063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 msgr2=0x56380edee1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380edee1d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f433c011e00 tx=0x7f433c004930 comp rx=0 tx=0).stop 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 -- 172.21.15.31:0/4013584063 shutdown_connections 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56380ed75a30 0x56380edee1d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56380ed6b170 0x56380ed78fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56380edeff90 0x56380edf2380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 --2- 172.21.15.31:0/4013584063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380ed6a850 0x56380ed78a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 -- 172.21.15.31:0/4013584063 >> 172.21.15.31:0/4013584063 conn(0x56380ed66de0 msgr2=0x56380ed719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f4329ffb640 0 client.15687 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 -- 172.21.15.31:0/4013584063 shutdown_connections 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 1 -- 172.21.15.31:0/4013584063 wait complete. 2024-09-16T00:43:47.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.261+0000 7f434e826740 20 client.15687 trim_cache size 0 max 16384 2024-09-16T00:43:47.263 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:43:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:43:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:43:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 10, 2024-09-16T00:43:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:43:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "33a4644b": { 2024-09-16T00:43:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 10, 2024-09-16T00:43:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 139.6, 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:43:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:43:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:43:47.265 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 -- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f432c0a3240 msgr2=0x7f432c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f432c0a3240 0x7f432c0a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4338009230 tx=0x7f4338004370 comp rx=0 tx=0).stop 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 -- 172.21.15.31:0/2998253269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4344100e20 msgr2=0x7f43440fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4344100e20 0x7f43440fcc80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f433400eb00 tx=0x7f433400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 -- 172.21.15.31:0/2998253269 shutdown_connections 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4344102ef0 0x7f43440fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4344102570 0x7f43440fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f432c0a3240 0x7f432c0a5700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 --2- 172.21.15.31:0/2998253269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4344100e20 0x7f43440fcc80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 -- 172.21.15.31:0/2998253269 >> 172.21.15.31:0/2998253269 conn(0x7f43440fc7d0 msgr2=0x7f434406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 -- 172.21.15.31:0/2998253269 shutdown_connections 2024-09-16T00:43:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:47.262+0000 7f434c44d640 1 -- 172.21.15.31:0/2998253269 wait complete. 2024-09-16T00:43:47.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:47 smithi031 ceph-mon[30956]: pgmap v435: 129 pgs: 129 active+clean; 237 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.5 MiB/s wr, 198 op/s 2024-09-16T00:43:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:47 smithi081 ceph-mon[35539]: pgmap v435: 129 pgs: 129 active+clean; 237 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.5 MiB/s wr, 198 op/s 2024-09-16T00:43:47.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:47 smithi102 ceph-mon[35364]: pgmap v435: 129 pgs: 129 active+clean; 237 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.5 MiB/s wr, 198 op/s 2024-09-16T00:43:48.763 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '33a4644b', quiesce took 11.0 seconds. Will release after: 8.55 2024-09-16T00:43:49.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:49 smithi031 ceph-mon[30956]: pgmap v436: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 3.4 MiB/s wr, 306 op/s 2024-09-16T00:43:49.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:49 smithi081 ceph-mon[35539]: pgmap v436: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 3.4 MiB/s wr, 306 op/s 2024-09-16T00:43:49.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:49 smithi102 ceph-mon[35364]: pgmap v436: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 3.4 MiB/s wr, 306 op/s 2024-09-16T00:43:51.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:51 smithi031 ceph-mon[30956]: pgmap v437: 129 pgs: 129 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 367 op/s 2024-09-16T00:43:51.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:51 smithi081 ceph-mon[35539]: pgmap v437: 129 pgs: 129 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 367 op/s 2024-09-16T00:43:51.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:51 smithi102 ceph-mon[35364]: pgmap v437: 129 pgs: 129 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 367 op/s 2024-09-16T00:43:53.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:53 smithi031 ceph-mon[30956]: pgmap v438: 129 pgs: 129 active+clean; 263 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-09-16T00:43:53.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:53 smithi081 ceph-mon[35539]: pgmap v438: 129 pgs: 129 active+clean; 263 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-09-16T00:43:53.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:53 smithi102 ceph-mon[35364]: pgmap v438: 129 pgs: 129 active+clean; 263 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-09-16T00:43:55.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:55 smithi081 ceph-mon[35539]: pgmap v439: 129 pgs: 129 active+clean; 263 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 419 op/s 2024-09-16T00:43:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:55 smithi102 ceph-mon[35364]: pgmap v439: 129 pgs: 129 active+clean; 263 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 419 op/s 2024-09-16T00:43:55.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:55 smithi031 ceph-mon[30956]: pgmap v439: 129 pgs: 129 active+clean; 263 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 419 op/s 2024-09-16T00:43:56.187 INFO:tasks.workunit.client.0.smithi031.stdout: 47 39868 0 24050 0 21652 0 2024-09-16T00:43:57.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:43:57.585 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:57.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:57 smithi031 ceph-mon[30956]: pgmap v440: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-09-16T00:43:57.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:57 smithi081 ceph-mon[35539]: pgmap v440: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-09-16T00:43:57.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:57 smithi102 ceph-mon[35364]: pgmap v440: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-09-16T00:43:57.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:43:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.104+0000 7f1698e7d640 1 -- 172.21.15.31:0/2173576989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1694100e20 msgr2=0x7f1694101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.104+0000 7f1698e7d640 1 --2- 172.21.15.31:0/2173576989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1694100e20 0x7f1694101200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f168400d940 tx=0x7f168402c440 comp rx=0 tx=0).stop 2024-09-16T00:43:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.104+0000 7f1698e7d640 1 -- 172.21.15.31:0/2173576989 shutdown_connections 2024-09-16T00:43:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.104+0000 7f1698e7d640 1 --2- 172.21.15.31:0/2173576989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1694102ef0 0x7f169410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.104+0000 7f1698e7d640 1 --2- 172.21.15.31:0/2173576989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694102570 0x7f16941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.104+0000 7f1698e7d640 1 --2- 172.21.15.31:0/2173576989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1694100e20 0x7f1694101200 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.104+0000 7f1698e7d640 1 -- 172.21.15.31:0/2173576989 >> 172.21.15.31:0/2173576989 conn(0x7f16940fc7d0 msgr2=0x7f16940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.104+0000 7f1698e7d640 1 -- 172.21.15.31:0/2173576989 shutdown_connections 2024-09-16T00:43:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.104+0000 7f1698e7d640 1 -- 172.21.15.31:0/2173576989 wait complete. 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.105+0000 7f1698e7d640 1 Processor -- start 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.105+0000 7f1698e7d640 1 -- start start 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.105+0000 7f1698e7d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1694100e20 0x7f169419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.105+0000 7f1698e7d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694102570 0x7f169419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.105+0000 7f1698e7d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1694102ef0 0x7f169419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.105+0000 7f1698e7d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f169410fe80 con 0x7f1694102570 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.105+0000 7f1698e7d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f169410fd00 con 0x7f1694100e20 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.105+0000 7f1698e7d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1694110000 con 0x7f1694102ef0 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f169259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1694100e20 0x7f169419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f169259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1694100e20 0x7f169419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:57174/0 (socket says 172.21.15.31:57174) 2024-09-16T00:43:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f169259c640 1 -- 172.21.15.31:0/1765745299 learned_addr learned my addr 172.21.15.31:0/1765745299 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f1691d9b640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694102570 0x7f169419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f1692d9d640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1694102ef0 0x7f169419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f169259c640 1 -- 172.21.15.31:0/1765745299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1694102ef0 msgr2=0x7f169419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f169259c640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1694102ef0 0x7f169419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f169259c640 1 -- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694102570 msgr2=0x7f169419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f169259c640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694102570 0x7f169419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f169259c640 1 -- 172.21.15.31:0/1765745299 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169419f6a0 con 0x7f1694100e20 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.106+0000 7f169259c640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1694100e20 0x7f169419a6f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1684000c00 tx=0x7f168400c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.107+0000 7f16837fe640 1 -- 172.21.15.31:0/1765745299 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840052d0 con 0x7f1694100e20 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.107+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169419f930 con 0x7f1694100e20 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.107+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1694107980 con 0x7f1694100e20 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.107+0000 7f16837fe640 1 -- 172.21.15.31:0/1765745299 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1684005470 con 0x7f1694100e20 2024-09-16T00:43:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.107+0000 7f16837fe640 1 -- 172.21.15.31:0/1765745299 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168400ccf0 con 0x7f1694100e20 2024-09-16T00:43:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.108+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1674005290 con 0x7f1694100e20 2024-09-16T00:43:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.109+0000 7f16837fe640 1 -- 172.21.15.31:0/1765745299 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1684004030 con 0x7f1694100e20 2024-09-16T00:43:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.110+0000 7f16837fe640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f167c0a3240 0x7f167c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.110+0000 7f16837fe640 1 -- 172.21.15.31:0/1765745299 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f16840f0a70 con 0x7f1694100e20 2024-09-16T00:43:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.110+0000 7f1691d9b640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f167c0a3240 0x7f167c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.111+0000 7f1691d9b640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f167c0a3240 0x7f167c0a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f169410de80 tx=0x7f1688005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.113+0000 7f16837fe640 1 -- 172.21.15.31:0/1765745299 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16840bd3c0 con 0x7f1694100e20 2024-09-16T00:43:58.243 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:43:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.357+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1674005580 con 0x7f1694100e20 2024-09-16T00:43:58.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.358+0000 7f16837fe640 1 -- 172.21.15.31:0/1765745299 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f16840bccc0 con 0x7f1694100e20 2024-09-16T00:43:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:43:58.360 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:43:58.360 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:43:58.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.360+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f167c0a3240 msgr2=0x7f167c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:58.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.360+0000 7f1698e7d640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f167c0a3240 0x7f167c0a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f169410de80 tx=0x7f1688005f90 comp rx=0 tx=0).stop 2024-09-16T00:43:58.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.360+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1694100e20 msgr2=0x7f169419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:58.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.360+0000 7f1698e7d640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1694100e20 0x7f169419a6f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1684000c00 tx=0x7f168400c500 comp rx=0 tx=0).stop 2024-09-16T00:43:58.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.361+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 shutdown_connections 2024-09-16T00:43:58.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.361+0000 7f1698e7d640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f167c0a3240 0x7f167c0a5700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:58.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.361+0000 7f1698e7d640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1694102ef0 0x7f169419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:58.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.361+0000 7f1698e7d640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694102570 0x7f169419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:58.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.361+0000 7f1698e7d640 1 --2- 172.21.15.31:0/1765745299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1694100e20 0x7f169419a6f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:58.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.361+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 >> 172.21.15.31:0/1765745299 conn(0x7f16940fc7d0 msgr2=0x7f16940fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:58.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.361+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 shutdown_connections 2024-09-16T00:43:58.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:58.361+0000 7f1698e7d640 1 -- 172.21.15.31:0/1765745299 wait complete. 2024-09-16T00:43:58.390 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:43:58.390 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:43:58.390 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:43:58.391 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:43:58.391 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:58 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1765745299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:43:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:43:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:43:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:43:58.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:58 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1765745299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:43:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:43:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:43:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:43:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:58 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1765745299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:43:59.273 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 33a4644b --release --await' 2024-09-16T00:43:59.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 33a4644b --release --await 2024-09-16T00:43:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.475+0000 7f8d008a0640 1 -- 172.21.15.31:0/3576287918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 msgr2=0x7f8cfc10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.475+0000 7f8d008a0640 1 --2- 172.21.15.31:0/3576287918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 0x7f8cfc10ec70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009cd0 tx=0x7f8cf402a8d0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.475+0000 7f8d008a0640 1 -- 172.21.15.31:0/3576287918 shutdown_connections 2024-09-16T00:43:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.475+0000 7f8d008a0640 1 --2- 172.21.15.31:0/3576287918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 0x7f8cfc10ec70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.475+0000 7f8d008a0640 1 --2- 172.21.15.31:0/3576287918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cfc103780 0x7f8cfc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.475+0000 7f8d008a0640 1 --2- 172.21.15.31:0/3576287918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8cfc102dd0 0x7f8cfc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.475+0000 7f8d008a0640 1 -- 172.21.15.31:0/3576287918 >> 172.21.15.31:0/3576287918 conn(0x7f8cfc0fc7d0 msgr2=0x7f8cfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.475+0000 7f8d008a0640 1 -- 172.21.15.31:0/3576287918 shutdown_connections 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.475+0000 7f8d008a0640 1 -- 172.21.15.31:0/3576287918 wait complete. 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8d008a0640 1 Processor -- start 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8d008a0640 1 -- start start 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8d008a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cfc102dd0 0x7f8cfc1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8d008a0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8cfc103780 0x7f8cfc1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8d008a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 0x7f8cfc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8d008a0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc112110 con 0x7f8cfc108300 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8d008a0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc111f90 con 0x7f8cfc102dd0 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8d008a0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc112290 con 0x7f8cfc103780 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cfad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 0x7f8cfc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cfad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 0x7f8cfc19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50784/0 (socket says 172.21.15.31:50784) 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cfad9d640 1 -- 172.21.15.31:0/495931348 learned_addr learned my addr 172.21.15.31:0/495931348 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8cfc103780 0x7f8cfc1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cfc102dd0 0x7f8cfc1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cfad9d640 1 -- 172.21.15.31:0/495931348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8cfc103780 msgr2=0x7f8cfc1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cfad9d640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8cfc103780 0x7f8cfc1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cfad9d640 1 -- 172.21.15.31:0/495931348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cfc102dd0 msgr2=0x7f8cfc1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cfad9d640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cfc102dd0 0x7f8cfc1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.476+0000 7f8cfad9d640 1 -- 172.21.15.31:0/495931348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc19ad40 con 0x7f8cfc108300 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.477+0000 7f8cfad9d640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 0x7f8cfc19a610 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009ca0 tx=0x7f8cf402f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.477+0000 7f8ceb7fe640 1 -- 172.21.15.31:0/495931348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40317a0 con 0x7f8cfc108300 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.477+0000 7f8d008a0640 1 -- 172.21.15.31:0/495931348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc19afd0 con 0x7f8cfc108300 2024-09-16T00:43:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.477+0000 7f8ceb7fe640 1 -- 172.21.15.31:0/495931348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf4003e50 con 0x7f8cfc108300 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.477+0000 7f8ceb7fe640 1 -- 172.21.15.31:0/495931348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4030590 con 0x7f8cfc108300 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.477+0000 7f8d008a0640 1 -- 172.21.15.31:0/495931348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc1a7320 con 0x7f8cfc108300 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.479+0000 7f8ceb7fe640 1 -- 172.21.15.31:0/495931348 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8cf4004160 con 0x7f8cfc108300 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.479+0000 7f8d02c79740 1 Processor -- start 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8d02c79740 1 -- start start 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3a6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 0x56535b3b1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8d02c79740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56535b30c2e0 con 0x56535b3a64a0 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8d02c79740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56535b2b3ce0 con 0x56535b3a6dc0 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8d02c79740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56535b3082a0 con 0x56535b3b1680 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8cf9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8cf9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50788/0 (socket says 172.21.15.31:50788) 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8cf9d9b640 1 -- 172.21.15.31:0/1559264185 learned_addr learned my addr 172.21.15.31:0/1559264185 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/1559264185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 0x56535b3b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.480+0000 7f8cf9d9b640 1 -- 172.21.15.31:0/1559264185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 msgr2=0x56535b3b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/1559264185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8cf9d9b640 1 -- 172.21.15.31:0/1559264185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 msgr2=0x56535b3b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/1559264185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 0x56535b3b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8cf9d9b640 1 -- 172.21.15.31:0/1559264185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56535b298440 con 0x56535b3a64a0 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8ceb7fe640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ce40a3240 0x7f8ce40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/1559264185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3a6880 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0697d0 tx=0x7f8cf00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd6c7da3e5bfa7d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ce40a3240 0x7f8ce40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8ceb7fe640 1 -- 172.21.15.31:0/495931348 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f8cf40f0e10 con 0x7f8cfc108300 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8ce8ff9640 1 -- 172.21.15.31:0/1559264185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0005020 con 0x56535b3a64a0 2024-09-16T00:43:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8ce8ff9640 1 -- 172.21.15.31:0/1559264185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf00047a0 con 0x56535b3a64a0 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.481+0000 7f8ce8ff9640 1 -- 172.21.15.31:0/1559264185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0012690 con 0x56535b3a64a0 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 -- 172.21.15.31:0/1559264185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 msgr2=0x56535b3a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ce40a3240 0x7f8ce40a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x56535b3a3890 tx=0x7f8cec009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 --2- 172.21.15.31:0/1559264185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3a6880 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0697d0 tx=0x7f8cf00117b0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 -- 172.21.15.31:0/1559264185 shutdown_connections 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 --2- 172.21.15.31:0/1559264185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 --2- 172.21.15.31:0/1559264185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 0x56535b3b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 --2- 172.21.15.31:0/1559264185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3a6880 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 -- 172.21.15.31:0/1559264185 >> 172.21.15.31:0/1559264185 conn(0x56535b3a2a30 msgr2=0x56535b3a2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 -- 172.21.15.31:0/1559264185 shutdown_connections 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 -- 172.21.15.31:0/1559264185 wait complete. 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 -- start start 2024-09-16T00:43:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.482+0000 7f8d02c79740 1 Processor -- start 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a64a0 0x56535b3b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a6dc0 0x56535b3ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8d02c79740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56535b3b4250 con 0x56535b3a6dc0 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8d02c79740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56535b2c7050 con 0x56535b3a64a0 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8d02c79740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56535b3b43d0 con 0x56535b3b1680 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8d02c79740 10 client.0 _ll_register_callbacks cb 0x56535b307160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cf9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a6dc0 0x56535b3ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cf9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a6dc0 0x56535b3ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50798/0 (socket says 172.21.15.31:50798) 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cf9d9b640 1 -- 172.21.15.31:0/2825286151 learned_addr learned my addr 172.21.15.31:0/2825286151 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cfad9d640 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a64a0 0x56535b3b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cf9d9b640 1 -- 172.21.15.31:0/2825286151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 msgr2=0x56535b3ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cf9d9b640 1 -- 172.21.15.31:0/2825286151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a64a0 msgr2=0x56535b3b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a64a0 0x56535b3b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cf9d9b640 1 -- 172.21.15.31:0/2825286151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56535b07d3c0 con 0x56535b3a6dc0 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a64a0 0x56535b3b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a6dc0 0x56535b3ae1e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00058c0 tx=0x7f8cf0004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2742336fb975984c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2825286151 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0005230 con 0x56535b3a6dc0 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2825286151 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f8cf00053d0 con 0x56535b3a6dc0 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.483+0000 7f8d02c79740 1 -- 172.21.15.31:0/2825286151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf00047a0 con 0x56535b3a6dc0 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2825286151 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8cf0012450 con 0x56535b3a6dc0 2024-09-16T00:43:59.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8d02c79740 10 client.15705 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8d02c79740 1 -- 172.21.15.31:0/2825286151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56535b3bfb70 con 0x56535b3a6dc0 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2825286151 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf00047a0 con 0x56535b3a6dc0 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2825286151 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f8cf0025490 con 0x56535b3a6dc0 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8d02c79740 10 client.15705 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8d02c79740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8d02c79740 20 client.15705 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56535b3c0f70 0x56535b3c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8d02c79740 4 client.15705 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8d02c79740 1 -- 172.21.15.31:0/2825286151 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56535b319850 con 0x56535b3c0f70 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.484+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56535b3c0f70 0x56535b3c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.485+0000 7f8ce1ffb640 10 client.15705 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.485+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56535b3c0f70 0x56535b3c3360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ca50d8bddce05b server_cookie=f956f144909b9d32 in_seq=0 out_seq=0 2024-09-16T00:43:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2825286151 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56535b319850 con 0x56535b3c0f70 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8ce1ffb640 10 client.15705 handle_command_reply: tid=0 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 client.15705 shutdown 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 -- 172.21.15.31:0/2825286151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a6dc0 msgr2=0x56535b3ae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a6dc0 0x56535b3ae1e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00058c0 tx=0x7f8cf0004b80 comp rx=0 tx=0).stop 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 -- 172.21.15.31:0/2825286151 shutdown_connections 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56535b3c0f70 0x56535b3c3360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a6dc0 0x56535b3ae1e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8ce1ffb640 0 client.15705 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2825286151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a64a0 0x56535b3b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 -- 172.21.15.31:0/2825286151 >> 172.21.15.31:0/2825286151 conn(0x56535b3a2a30 msgr2=0x56535b3a8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 -- 172.21.15.31:0/2825286151 shutdown_connections 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 1 -- 172.21.15.31:0/2825286151 wait complete. 2024-09-16T00:43:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.486+0000 7f8d02c79740 20 client.15705 trim_cache size 0 max 16384 2024-09-16T00:43:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.503+0000 7f8d02c79740 1 Processor -- start 2024-09-16T00:43:59.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:43:59 smithi031 ceph-mon[30956]: pgmap v441: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 331 op/s 2024-09-16T00:43:59.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.503+0000 7f8d02c79740 1 -- start start 2024-09-16T00:43:59.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.503+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3a6dc0 0x56535b3c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3b1680 0x56535b3c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8d02c79740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1aeed0 con 0x56535b3a64a0 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8d02c79740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1b1750 con 0x56535b3b1680 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8d02c79740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1af540 con 0x56535b3a6dc0 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3bd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50802/0 (socket says 172.21.15.31:50802) 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfa59c640 1 -- 172.21.15.31:0/3857668186 learned_addr learned my addr 172.21.15.31:0/3857668186 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfad9d640 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3b1680 0x56535b3c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3a6dc0 0x56535b3c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfa59c640 1 -- 172.21.15.31:0/3857668186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3a6dc0 msgr2=0x56535b3c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3a6dc0 0x56535b3c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfa59c640 1 -- 172.21.15.31:0/3857668186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3b1680 msgr2=0x56535b3c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3b1680 0x56535b3c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfa59c640 1 -- 172.21.15.31:0/3857668186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56535b3c36d0 con 0x56535b3a64a0 2024-09-16T00:43:59.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfad9d640 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3b1680 0x56535b3c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:59.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.504+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3bd4b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8cec011cf0 tx=0x7f8cec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc5944f2c0355b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:59.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.505+0000 7f8ce3fff640 1 -- 172.21.15.31:0/3857668186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec033070 con 0x56535b3a64a0 2024-09-16T00:43:59.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.505+0000 7f8ce3fff640 1 -- 172.21.15.31:0/3857668186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cec01b040 con 0x56535b3a64a0 2024-09-16T00:43:59.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.505+0000 7f8ce3fff640 1 -- 172.21.15.31:0/3857668186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec014c20 con 0x56535b3a64a0 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 -- 172.21.15.31:0/3857668186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 msgr2=0x56535b3bd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3bd4b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8cec011cf0 tx=0x7f8cec014000 comp rx=0 tx=0).stop 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 -- 172.21.15.31:0/3857668186 shutdown_connections 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3b1680 0x56535b3c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3a6dc0 0x56535b3c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 --2- 172.21.15.31:0/3857668186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3bd4b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 -- 172.21.15.31:0/3857668186 >> 172.21.15.31:0/3857668186 conn(0x56535b3a2a30 msgr2=0x56535b3ad2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 -- 172.21.15.31:0/3857668186 shutdown_connections 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 -- 172.21.15.31:0/3857668186 wait complete. 2024-09-16T00:43:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.506+0000 7f8d02c79740 1 -- start start 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8d02c79740 1 Processor -- start 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 0x56535b3b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8d02c79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8d02c79740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1afc10 con 0x56535b3a64a0 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3b9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50806/0 (socket says 172.21.15.31:50806) 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8cfa59c640 1 -- 172.21.15.31:0/2250576070 learned_addr learned my addr 172.21.15.31:0/2250576070 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1b1080 con 0x56535b3a6dc0 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1b02e0 con 0x56535b3b1680 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 0x56535b3b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8cfad9d640 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.507+0000 7f8d02c79740 10 client.0 _ll_register_callbacks cb 0x7f8cec0093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8cfa59c640 1 -- 172.21.15.31:0/2250576070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 msgr2=0x56535b3b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8cfa59c640 1 -- 172.21.15.31:0/2250576070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 msgr2=0x56535b3b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 0x56535b3b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8cfa59c640 1 -- 172.21.15.31:0/2250576070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56535b3b72a0 con 0x56535b3a64a0 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8cfad9d640 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 0x56535b3b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8cfa59c640 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3b9640 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8cec011690 tx=0x7f8cec014440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e79b084d483212 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2250576070 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec03e060 con 0x56535b3a64a0 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2250576070 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f8cec029040 con 0x56535b3a64a0 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cec01b040 con 0x56535b3a64a0 2024-09-16T00:43:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.508+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2250576070 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8cec033030 con 0x56535b3a64a0 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8d02c79740 10 client.15711 fetch_fsmap learned FSMap version 20 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56535b42b640 con 0x56535b3a64a0 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2250576070 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cec01b040 con 0x56535b3a64a0 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2250576070 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f8cec039560 con 0x56535b3a64a0 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8d02c79740 10 client.15711 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8d02c79740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8d02c79740 20 client.15711 populate_metadata read hostname 'smithi031' 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56535b42bcd0 0x56535b42e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8d02c79740 4 client.15711 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56535b313d60 con 0x56535b42bcd0 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.509+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56535b42bcd0 0x56535b42e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8cf9d9b640 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56535b42bcd0 0x56535b42e0f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c19256a1d7f017 server_cookie=bd1d3fada20d4179 in_seq=0 out_seq=0 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8ce1ffb640 10 client.15711 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:59.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8ce1ffb640 1 -- 172.21.15.31:0/2250576070 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56535b313d60 con 0x56535b42bcd0 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8ce1ffb640 10 client.15711 handle_command_reply: tid=0 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 client.15711 shutdown 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 msgr2=0x56535b3b9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3b9640 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8cec011690 tx=0x7f8cec014440 comp rx=0 tx=0).stop 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 shutdown_connections 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56535b42bcd0 0x56535b42e0f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56535b3b1680 0x56535b3b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56535b3a6dc0 0x56535b3b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 --2- 172.21.15.31:0/2250576070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56535b3a64a0 0x56535b3b9640 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 >> 172.21.15.31:0/2250576070 conn(0x56535b3a2a30 msgr2=0x56535b3afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8ce1ffb640 0 client.15711 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:43:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 shutdown_connections 2024-09-16T00:43:59.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.510+0000 7f8d02c79740 1 -- 172.21.15.31:0/2250576070 wait complete. 2024-09-16T00:43:59.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.511+0000 7f8d02c79740 20 client.15711 trim_cache size 0 max 16384 2024-09-16T00:43:59.517 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:43:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:43:59.517 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:43:59.517 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.511+0000 7f8d008a0640 1 -- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ce40a3240 msgr2=0x7f8ce40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.511+0000 7f8d008a0640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ce40a3240 0x7f8ce40a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x56535b3a3890 tx=0x7f8cec009110 comp rx=0 tx=0).stop 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.511+0000 7f8d008a0640 1 -- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 msgr2=0x7f8cfc19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.511+0000 7f8d008a0640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 0x7f8cfc19a610 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009ca0 tx=0x7f8cf402f420 comp rx=0 tx=0).stop 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.512+0000 7f8d008a0640 1 -- 172.21.15.31:0/495931348 shutdown_connections 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.512+0000 7f8d008a0640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cfc108300 0x7f8cfc19a610 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.512+0000 7f8d008a0640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8ce40a3240 0x7f8ce40a5700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.512+0000 7f8d008a0640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8cfc103780 0x7f8cfc1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.512+0000 7f8d008a0640 1 --2- 172.21.15.31:0/495931348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cfc102dd0 0x7f8cfc1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.512+0000 7f8d008a0640 1 -- 172.21.15.31:0/495931348 >> 172.21.15.31:0/495931348 conn(0x7f8cfc0fc7d0 msgr2=0x7f8cfc0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.512+0000 7f8d008a0640 1 -- 172.21.15.31:0/495931348 shutdown_connections 2024-09-16T00:43:59.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:43:59.512+0000 7f8d008a0640 1 -- 172.21.15.31:0/495931348 wait complete. 2024-09-16T00:43:59.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:43:59 smithi081 ceph-mon[35539]: pgmap v441: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 331 op/s 2024-09-16T00:43:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:43:59 smithi102 ceph-mon[35364]: pgmap v441: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 331 op/s 2024-09-16T00:44:00.433 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:44:00.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:44:00.439 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 -- 172.21.15.31:0/1685747360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a8101280 msgr2=0x7fd9a810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/1685747360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a8101280 0x7fd9a810df40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd998009900 tx=0x7fd99802a410 comp rx=0 tx=0).stop 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 -- 172.21.15.31:0/1685747360 shutdown_connections 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/1685747360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9a810e480 0x7fd9a8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/1685747360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a8101280 0x7fd9a810df40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/1685747360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a81008d0 0x7fd9a8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 -- 172.21.15.31:0/1685747360 >> 172.21.15.31:0/1685747360 conn(0x7fd9a80fc7a0 msgr2=0x7fd9a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 -- 172.21.15.31:0/1685747360 shutdown_connections 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 -- 172.21.15.31:0/1685747360 wait complete. 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 Processor -- start 2024-09-16T00:44:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.938+0000 7fd9af8cd640 1 -- start start 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9af8cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a81008d0 0x7fd9a819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9af8cd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9a8101280 0x7fd9a819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9af8cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a810e480 0x7fd9a819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9af8cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81141e0 con 0x7fd9a81008d0 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9af8cd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8114060 con 0x7fd9a8101280 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9af8cd640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8114360 con 0x7fd9a810e480 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ad669640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a81008d0 0x7fd9a819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ad669640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a81008d0 0x7fd9a819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50814/0 (socket says 172.21.15.31:50814) 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ad669640 1 -- 172.21.15.31:0/3823740935 learned_addr learned my addr 172.21.15.31:0/3823740935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a810e480 0x7fd9a819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ace68640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9a8101280 0x7fd9a819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ad669640 1 -- 172.21.15.31:0/3823740935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a810e480 msgr2=0x7fd9a819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ad669640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a810e480 0x7fd9a819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ad669640 1 -- 172.21.15.31:0/3823740935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9a8101280 msgr2=0x7fd9a819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ad669640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9a8101280 0x7fd9a819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ad669640 1 -- 172.21.15.31:0/3823740935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a819f670 con 0x7fd9a81008d0 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9ad669640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a81008d0 0x7fd9a819a6c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400d950 tx=0x7fd9a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd99e7fc640 1 -- 172.21.15.31:0/3823740935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4017070 con 0x7fd9a81008d0 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd9af8cd640 1 -- 172.21.15.31:0/3823740935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a819f960 con 0x7fd9a81008d0 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd99e7fc640 1 -- 172.21.15.31:0/3823740935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a4009c30 con 0x7fd9a81008d0 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.939+0000 7fd99e7fc640 1 -- 172.21.15.31:0/3823740935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400f6b0 con 0x7fd9a81008d0 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.940+0000 7fd9af8cd640 1 -- 172.21.15.31:0/3823740935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a81a7240 con 0x7fd9a81008d0 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.942+0000 7fd99e7fc640 1 -- 172.21.15.31:0/3823740935 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd9a40109d0 con 0x7fd9a81008d0 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.942+0000 7fd9b1ca6740 1 Processor -- start 2024-09-16T00:44:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.942+0000 7fd9b1ca6740 1 -- start start 2024-09-16T00:44:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x561982300630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561982258620 con 0x5619822f3060 2024-09-16T00:44:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5619821ffe50 con 0x5619822f3980 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5619822545e0 con 0x5619822fe240 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ade6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ade6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50816/0 (socket says 172.21.15.31:50816) 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ade6a640 1 -- 172.21.15.31:0/460005720 learned_addr learned my addr 172.21.15.31:0/460005720 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd99e7fc640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd9940a3240 0x7fd9940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ad669640 1 --2- 172.21.15.31:0/460005720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ace68640 1 --2- 172.21.15.31:0/460005720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x561982300630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd9940a3240 0x7fd9940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ade6a640 1 -- 172.21.15.31:0/460005720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 msgr2=0x561982300630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/460005720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x561982300630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd99e7fc640 1 -- 172.21.15.31:0/3823740935 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd9a40cf990 con 0x7fd9a81008d0 2024-09-16T00:44:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ade6a640 1 -- 172.21.15.31:0/460005720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 msgr2=0x5619822fdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/460005720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822fdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.943+0000 7fd9ade6a640 1 -- 172.21.15.31:0/460005720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619821f0d40 con 0x5619822f3060 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.944+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/460005720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f3440 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00049d0 tx=0x7fd9a0014eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3779d070dee0cfde server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.944+0000 7fd9937fe640 1 -- 172.21.15.31:0/460005720 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a001ea80 con 0x5619822f3060 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.944+0000 7fd9937fe640 1 -- 172.21.15.31:0/460005720 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a0023070 con 0x5619822f3060 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.944+0000 7fd9937fe640 1 -- 172.21.15.31:0/460005720 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000fca0 con 0x5619822f3060 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.944+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd9940a3240 0x7fd9940a5700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0022000 tx=0x7fd9a0020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.944+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/460005720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 msgr2=0x5619822f3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.944+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/460005720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f3440 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00049d0 tx=0x7fd9a0014eb0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/460005720 shutdown_connections 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/460005720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x561982300630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/460005720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822fdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/460005720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f3440 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/460005720 >> 172.21.15.31:0/460005720 conn(0x5619822ef5f0 msgr2=0x5619822ef9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/460005720 shutdown_connections 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/460005720 wait complete. 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 -- start start 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 Processor -- start 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822fb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x5619822fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561982300e10 con 0x5619822f3060 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5619821e0f10 con 0x5619822f3980 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.945+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561982300f90 con 0x5619822fe240 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9b1ca6740 10 client.0 _ll_register_callbacks cb 0x561982204db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:00.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ad669640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ad669640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822fb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50822/0 (socket says 172.21.15.31:50822) 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ad669640 1 -- 172.21.15.31:0/2814327716 learned_addr learned my addr 172.21.15.31:0/2814327716 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x5619822fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ace68640 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ad669640 1 -- 172.21.15.31:0/2814327716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 msgr2=0x5619822fbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ad669640 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x5619822fbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ad669640 1 -- 172.21.15.31:0/2814327716 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 msgr2=0x5619822fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ad669640 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ad669640 1 -- 172.21.15.31:0/2814327716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561981f67580 con 0x5619822f3060 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9ad669640 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822fb0f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd9a409bb30 tx=0x7fd9a400dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61aed0d2b19da83 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:44:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd990ff9640 1 -- 172.21.15.31:0/2814327716 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4020040 con 0x5619822f3060 2024-09-16T00:44:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2814327716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a0023070 con 0x5619822f3060 2024-09-16T00:44:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd990ff9640 1 -- 172.21.15.31:0/2814327716 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd9a40860e0 con 0x5619822f3060 2024-09-16T00:44:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd990ff9640 1 -- 172.21.15.31:0/2814327716 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd9a409bba0 con 0x5619822f3060 2024-09-16T00:44:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9b1ca6740 10 client.15723 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.946+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2814327716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561982031960 con 0x5619822f3060 2024-09-16T00:44:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.947+0000 7fd990ff9640 1 -- 172.21.15.31:0/2814327716 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a0023070 con 0x5619822f3060 2024-09-16T00:44:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.947+0000 7fd990ff9640 1 -- 172.21.15.31:0/2814327716 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd9a4096340 con 0x5619822f3060 2024-09-16T00:44:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.947+0000 7fd9b1ca6740 10 client.15723 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.947+0000 7fd9b1ca6740 10 client.15723 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:44:00.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.947+0000 7fd9b1ca6740 20 client.15723 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:00.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.947+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56198230de30 0x561982310220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.947+0000 7fd9b1ca6740 4 client.15723 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.947+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2814327716 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561982265b90 con 0x56198230de30 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.947+0000 7fd9ace68640 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56198230de30 0x561982310220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd990ff9640 10 client.15723 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9ace68640 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56198230de30 0x561982310220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d2745dc8653d81 server_cookie=8bb9e4d503d77f70 in_seq=0 out_seq=0 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd990ff9640 1 -- 172.21.15.31:0/2814327716 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561982265b90 con 0x56198230de30 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd990ff9640 10 client.15723 handle_command_reply: tid=0 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9b1ca6740 1 client.15723 shutdown 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2814327716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 msgr2=0x5619822fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822fb0f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd9a409bb30 tx=0x7fd9a400dfa0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2814327716 shutdown_connections 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56198230de30 0x561982310220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x5619822fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd990ff9640 0 client.15723 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:00.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2814327716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822fb0f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.948+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2814327716 >> 172.21.15.31:0/2814327716 conn(0x5619822ef5f0 msgr2=0x5619822ee750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:00.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.949+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2814327716 shutdown_connections 2024-09-16T00:44:00.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.949+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2814327716 wait complete. 2024-09-16T00:44:00.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.949+0000 7fd9b1ca6740 20 client.15723 trim_cache size 0 max 16384 2024-09-16T00:44:00.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9b1ca6740 1 Processor -- start 2024-09-16T00:44:00.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9b1ca6740 1 -- start start 2024-09-16T00:44:00.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822ee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x561982315a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81aeed0 con 0x5619822f3060 2024-09-16T00:44:00.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81b1750 con 0x5619822f3980 2024-09-16T00:44:00.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81af540 con 0x5619822fe240 2024-09-16T00:44:00.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9ad669640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9ad669640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50824/0 (socket says 172.21.15.31:50824) 2024-09-16T00:44:00.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9ad669640 1 -- 172.21.15.31:0/165270424 learned_addr learned my addr 172.21.15.31:0/165270424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:00.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9ace68640 1 --2- 172.21.15.31:0/165270424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/165270424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x561982315a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9ad669640 1 -- 172.21.15.31:0/165270424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 msgr2=0x561982315a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:00.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.966+0000 7fd9ad669640 1 --2- 172.21.15.31:0/165270424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x561982315a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.967+0000 7fd9ad669640 1 -- 172.21.15.31:0/165270424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 msgr2=0x5619822ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.967+0000 7fd9ad669640 1 --2- 172.21.15.31:0/165270424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.967+0000 7fd9ad669640 1 -- 172.21.15.31:0/165270424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561982310850 con 0x5619822f3060 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.967+0000 7fd9ad669640 1 --2- 172.21.15.31:0/165270424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f5ac0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40058c0 tx=0x7fd9a4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf292ffab5b518c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.967+0000 7fd992ffd640 1 -- 172.21.15.31:0/165270424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4020070 con 0x5619822f3060 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.967+0000 7fd992ffd640 1 -- 172.21.15.31:0/165270424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a40a4810 con 0x5619822f3060 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.967+0000 7fd992ffd640 1 -- 172.21.15.31:0/165270424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a409b0c0 con 0x5619822f3060 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/165270424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 msgr2=0x5619822f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/165270424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f5ac0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40058c0 tx=0x7fd9a4010d70 comp rx=0 tx=0).stop 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/165270424 shutdown_connections 2024-09-16T00:44:00.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/165270424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822fe240 0x561982315a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/165270424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822f3980 0x5619822ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/165270424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x5619822f5ac0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/165270424 >> 172.21.15.31:0/165270424 conn(0x5619822ef5f0 msgr2=0x5619822edf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/165270424 shutdown_connections 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/165270424 wait complete. 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.968+0000 7fd9b1ca6740 1 -- start start 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9b1ca6740 1 Processor -- start 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x561982302870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822f3980 0x561982302db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9b1ca6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822fe240 0x561982371a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81afc10 con 0x5619822f3060 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81b1080 con 0x5619822fe240 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9b1ca6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81b02e0 con 0x5619822f3980 2024-09-16T00:44:00.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9b1ca6740 10 client.0 _ll_register_callbacks cb 0x7fd9a409ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9ad669640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x561982302870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9ad669640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x561982302870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50830/0 (socket says 172.21.15.31:50830) 2024-09-16T00:44:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9ad669640 1 -- 172.21.15.31:0/2198624790 learned_addr learned my addr 172.21.15.31:0/2198624790 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.969+0000 7fd9ace68640 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822f3980 0x561982302db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822fe240 0x561982371a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9ad669640 1 -- 172.21.15.31:0/2198624790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822f3980 msgr2=0x561982302db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9ad669640 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822f3980 0x561982302db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9ad669640 1 -- 172.21.15.31:0/2198624790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822fe240 msgr2=0x561982371a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9ad669640 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822fe240 0x561982371a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9ad669640 1 -- 172.21.15.31:0/2198624790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619823720c0 con 0x5619822f3060 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9ade6a640 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822fe240 0x561982371a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9ad669640 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x561982302870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009610 tx=0x7fd9a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b20954d63d32490 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd990ff9640 1 -- 172.21.15.31:0/2198624790 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4020070 con 0x5619822f3060 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd990ff9640 1 -- 172.21.15.31:0/2198624790 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd9a4096390 con 0x5619822f3060 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2198624790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a40a4810 con 0x5619822f3060 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd990ff9640 1 -- 172.21.15.31:0/2198624790 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd9a409ff50 con 0x5619822f3060 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9b1ca6740 10 client.15729 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.970+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2198624790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5619823720c0 con 0x5619822f3060 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.971+0000 7fd990ff9640 1 -- 172.21.15.31:0/2198624790 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a40a4810 con 0x5619822f3060 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.971+0000 7fd990ff9640 1 -- 172.21.15.31:0/2198624790 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd9a4010460 con 0x5619822f3060 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.971+0000 7fd9b1ca6740 10 client.15729 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.971+0000 7fd9b1ca6740 10 client.15729 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.971+0000 7fd9b1ca6740 20 client.15729 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.971+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5619823797b0 0x56198237bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.971+0000 7fd9b1ca6740 4 client.15729 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "33a4644b", "release": true, "await": true}] 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.971+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2198624790 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "set_id": "33a4644b", "release": true, "await": true}) -- 0x5619822600a0 con 0x5619823797b0 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.972+0000 7fd9ace68640 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5619823797b0 0x56198237bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.972+0000 7fd990ff9640 10 client.15729 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:00.972+0000 7fd9ace68640 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5619823797b0 0x56198237bba0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e231c02bd7bb893a server_cookie=46abdd996014a850 in_seq=0 out_seq=0 2024-09-16T00:44:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:01.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.406+0000 7fd990ff9640 1 -- 172.21.15.31:0/2198624790 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5619822600a0 con 0x5619823797b0 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.406+0000 7fd990ff9640 10 client.15729 handle_command_reply: tid=0 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.406+0000 7fd9b1ca6740 1 client.15729 shutdown 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2198624790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 msgr2=0x561982302870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x561982302870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009610 tx=0x7fd9a4005f90 comp rx=0 tx=0).stop 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2198624790 shutdown_connections 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5619822fe240 0x561982371a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5619823797b0 0x56198237bba0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619822f3980 0x561982302db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 --2- 172.21.15.31:0/2198624790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619822f3060 0x561982302870 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2198624790 >> 172.21.15.31:0/2198624790 conn(0x5619822ef5f0 msgr2=0x5619822fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd990ff9640 0 client.15729 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:01.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2198624790 shutdown_connections 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 1 -- 172.21.15.31:0/2198624790 wait complete. 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.407+0000 7fd9b1ca6740 20 client.15729 trim_cache size 0 max 16384 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 12, 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stdout: "33a4644b": { 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 12, 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:44:01.410 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 139.6, 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:44:01.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:44:01.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:44:01.412 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:44:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 -- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd9940a3240 msgr2=0x7fd9940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd9940a3240 0x7fd9940a5700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0022000 tx=0x7fd9a0020000 comp rx=0 tx=0).stop 2024-09-16T00:44:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 -- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a81008d0 msgr2=0x7fd9a819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a81008d0 0x7fd9a819a6c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400d950 tx=0x7fd9a400de10 comp rx=0 tx=0).stop 2024-09-16T00:44:01.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 -- 172.21.15.31:0/3823740935 shutdown_connections 2024-09-16T00:44:01.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd9940a3240 0x7fd9940a5700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a810e480 0x7fd9a819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9a8101280 0x7fd9a819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 --2- 172.21.15.31:0/3823740935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9a81008d0 0x7fd9a819a6c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 -- 172.21.15.31:0/3823740935 >> 172.21.15.31:0/3823740935 conn(0x7fd9a80fc7a0 msgr2=0x7fd9a80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:01.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 -- 172.21.15.31:0/3823740935 shutdown_connections 2024-09-16T00:44:01.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.408+0000 7fd9af8cd640 1 -- 172.21.15.31:0/3823740935 wait complete. 2024-09-16T00:44:01.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.487+0000 7f603b972640 1 -- 172.21.15.31:0/2694089971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034106f10 msgr2=0x7f603410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.487+0000 7f603b972640 1 --2- 172.21.15.31:0/2694089971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034106f10 0x7f603410ea90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f602c009ca0 tx=0x7f602c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.487+0000 7f603b972640 1 -- 172.21.15.31:0/2694089971 shutdown_connections 2024-09-16T00:44:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.487+0000 7f603b972640 1 --2- 172.21.15.31:0/2694089971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034106f10 0x7f603410ea90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.487+0000 7f603b972640 1 --2- 172.21.15.31:0/2694089971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6034102790 0x7f60341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.487+0000 7f603b972640 1 --2- 172.21.15.31:0/2694089971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6034101de0 0x7f60341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.487+0000 7f603b972640 1 -- 172.21.15.31:0/2694089971 >> 172.21.15.31:0/2694089971 conn(0x7f60340fd8c0 msgr2=0x7f60340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.487+0000 7f603b972640 1 -- 172.21.15.31:0/2694089971 shutdown_connections 2024-09-16T00:44:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.487+0000 7f603b972640 1 -- 172.21.15.31:0/2694089971 wait complete. 2024-09-16T00:44:01.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603b972640 1 Processor -- start 2024-09-16T00:44:01.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603b972640 1 -- start start 2024-09-16T00:44:01.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603b972640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034101de0 0x7f6034198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:01.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603b972640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6034102790 0x7f6034198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603b972640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6034106f10 0x7f603419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603b972640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6034111f70 con 0x7f6034101de0 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603b972640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6034111df0 con 0x7f6034102790 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603b972640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f60341120f0 con 0x7f6034106f10 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603970e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034101de0 0x7f6034198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603970e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034101de0 0x7f6034198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50852/0 (socket says 172.21.15.31:50852) 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f603970e640 1 -- 172.21.15.31:0/104479546 learned_addr learned my addr 172.21.15.31:0/104479546 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.488+0000 7f6038f0d640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6034102790 0x7f6034198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f6039f0f640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6034106f10 0x7f603419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f603970e640 1 -- 172.21.15.31:0/104479546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6034106f10 msgr2=0x7f603419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f603970e640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6034106f10 0x7f603419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f603970e640 1 -- 172.21.15.31:0/104479546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6034102790 msgr2=0x7f6034198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f603970e640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6034102790 0x7f6034198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f603970e640 1 -- 172.21.15.31:0/104479546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603419d2e0 con 0x7f6034101de0 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f603970e640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034101de0 0x7f6034198330 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f603000c920 tx=0x7f603000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f602a7fc640 1 -- 172.21.15.31:0/104479546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030007c70 con 0x7f6034101de0 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f602a7fc640 1 -- 172.21.15.31:0/104479546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6030007e10 con 0x7f6034101de0 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603419d5d0 con 0x7f6034101de0 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f602a7fc640 1 -- 172.21.15.31:0/104479546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030018890 con 0x7f6034101de0 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.489+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60341a4eb0 con 0x7f6034101de0 2024-09-16T00:44:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.490+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60341021c0 con 0x7f6034101de0 2024-09-16T00:44:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.492+0000 7f602a7fc640 1 -- 172.21.15.31:0/104479546 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6030018a50 con 0x7f6034101de0 2024-09-16T00:44:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.493+0000 7f602a7fc640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60200a3240 0x7f60200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.493+0000 7f6038f0d640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60200a3240 0x7f60200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:01.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.493+0000 7f602a7fc640 1 -- 172.21.15.31:0/104479546 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f60300d00e0 con 0x7f6034101de0 2024-09-16T00:44:01.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.494+0000 7f6038f0d640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60200a3240 0x7f60200a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6034108880 tx=0x7f6024005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:01.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.497+0000 7f602a7fc640 1 -- 172.21.15.31:0/104479546 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6030014030 con 0x7f6034101de0 2024-09-16T00:44:01.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:01 smithi031 ceph-mon[30956]: pgmap v442: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-09-16T00:44:01.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.728+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6034199690 con 0x7f6034101de0 2024-09-16T00:44:01.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.729+0000 7f602a7fc640 1 -- 172.21.15.31:0/104479546 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f603009cae0 con 0x7f6034101de0 2024-09-16T00:44:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:44:01.730 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:44:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:44:01.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60200a3240 msgr2=0x7f60200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60200a3240 0x7f60200a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6034108880 tx=0x7f6024005f90 comp rx=0 tx=0).stop 2024-09-16T00:44:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034101de0 msgr2=0x7f6034198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034101de0 0x7f6034198330 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f603000c920 tx=0x7f603000cde0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 shutdown_connections 2024-09-16T00:44:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60200a3240 0x7f60200a5700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6034106f10 0x7f603419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6034102790 0x7f6034198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 --2- 172.21.15.31:0/104479546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6034101de0 0x7f6034198330 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 >> 172.21.15.31:0/104479546 conn(0x7f60340fd8c0 msgr2=0x7f60340febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:01.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 shutdown_connections 2024-09-16T00:44:01.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:01.731+0000 7f603b972640 1 -- 172.21.15.31:0/104479546 wait complete. 2024-09-16T00:44:01.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:01 smithi081 ceph-mon[35539]: pgmap v442: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-09-16T00:44:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:01 smithi102 ceph-mon[35364]: pgmap v442: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-09-16T00:44:02.247 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '33a4644b', total seconds elapsed: 24.5 2024-09-16T00:44:02.597 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:02 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/104479546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:02.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:02 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/104479546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:02.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:02 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/104479546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:44:02.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:44:03.217 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:03.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:03 smithi031 ceph-mon[30956]: pgmap v443: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 500 KiB/s wr, 170 op/s 2024-09-16T00:44:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.740+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/3936599346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 msgr2=0x7f8b64101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.740+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/3936599346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 0x7f8b64101200 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009900 tx=0x7f8b5802a480 comp rx=0 tx=0).stop 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.740+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/3936599346 shutdown_connections 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.740+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/3936599346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b64102ef0 0x7f8b6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.740+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/3936599346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b64102570 0x7f8b641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.740+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/3936599346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 0x7f8b64101200 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.740+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/3936599346 >> 172.21.15.31:0/3936599346 conn(0x7f8b640fc7d0 msgr2=0x7f8b640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.740+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/3936599346 shutdown_connections 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.740+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/3936599346 wait complete. 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b6ad6f640 1 Processor -- start 2024-09-16T00:44:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b6ad6f640 1 -- start start 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b6ad6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 0x7f8b640fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b6ad6f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b64102570 0x7f8b640fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b6ad6f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b64102ef0 0x7f8b640fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b6ad6f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64110fd0 con 0x7f8b64100e20 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b6ad6f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64110e50 con 0x7f8b64102570 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b6ad6f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64111150 con 0x7f8b64102ef0 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b68b0b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 0x7f8b640fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b68b0b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 0x7f8b640fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50878/0 (socket says 172.21.15.31:50878) 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b68b0b640 1 -- 172.21.15.31:0/4123750840 learned_addr learned my addr 172.21.15.31:0/4123750840 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b63fff640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b64102570 0x7f8b640fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b6930c640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b64102ef0 0x7f8b640fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b68b0b640 1 -- 172.21.15.31:0/4123750840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b64102ef0 msgr2=0x7f8b640fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b64102ef0 0x7f8b640fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b68b0b640 1 -- 172.21.15.31:0/4123750840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b64102570 msgr2=0x7f8b640fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b64102570 0x7f8b640fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b68b0b640 1 -- 172.21.15.31:0/4123750840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b641a69a0 con 0x7f8b64100e20 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.741+0000 7f8b63fff640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b64102570 0x7f8b640fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.742+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 0x7f8b640fcc80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8b5802aa70 tx=0x7f8b58003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.742+0000 7f8b61ffb640 1 -- 172.21.15.31:0/4123750840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5803f070 con 0x7f8b64100e20 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.742+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/4123750840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b641a6c30 con 0x7f8b64100e20 2024-09-16T00:44:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.742+0000 7f8b61ffb640 1 -- 172.21.15.31:0/4123750840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b58031c30 con 0x7f8b64100e20 2024-09-16T00:44:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.742+0000 7f8b61ffb640 1 -- 172.21.15.31:0/4123750840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58035030 con 0x7f8b64100e20 2024-09-16T00:44:03.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.742+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/4123750840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b641a7160 con 0x7f8b64100e20 2024-09-16T00:44:03.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:03 smithi081 ceph-mon[35539]: pgmap v443: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 500 KiB/s wr, 170 op/s 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.745+0000 7f8b61ffb640 1 -- 172.21.15.31:0/4123750840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8b58042070 con 0x7f8b64100e20 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.745+0000 7f8b61ffb640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8b540a3240 0x7f8b540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.745+0000 7f8b6d148740 1 Processor -- start 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6d148740 1 -- start start 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b63fff640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8b540a3240 0x7f8b540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b61ffb640 1 -- 172.21.15.31:0/4123750840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f8b580f03e0 con 0x7f8b64100e20 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 0x55db44a79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a795d0 0x55db44a79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6d148740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55db449df2f0 con 0x55db44a84370 2024-09-16T00:44:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6d148740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55db44986cf0 con 0x55db44a795d0 2024-09-16T00:44:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6d148740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55db449db2b0 con 0x55db44a7ab40 2024-09-16T00:44:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6930c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6930c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a86760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50890/0 (socket says 172.21.15.31:50890) 2024-09-16T00:44:03.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6930c640 1 -- 172.21.15.31:0/705465836 learned_addr learned my addr 172.21.15.31:0/705465836 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:03.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b63fff640 1 --2- 172.21.15.31:0/705465836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a795d0 0x55db44a79a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/705465836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 0x55db44a79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6930c640 1 -- 172.21.15.31:0/705465836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 msgr2=0x55db44a79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:03.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6930c640 1 --2- 172.21.15.31:0/705465836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 0x55db44a79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6930c640 1 -- 172.21.15.31:0/705465836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a795d0 msgr2=0x55db44a79a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6930c640 1 --2- 172.21.15.31:0/705465836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a795d0 0x55db44a79a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b6930c640 1 -- 172.21.15.31:0/705465836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db4496b3a0 con 0x55db44a84370 2024-09-16T00:44:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.746+0000 7f8b63fff640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8b540a3240 0x7f8b540a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b500095f0 tx=0x7f8b50005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.747+0000 7f8b6930c640 1 --2- 172.21.15.31:0/705465836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a86760 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009e60 tx=0x7f8b5c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78ba19b62130918 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.747+0000 7f8b4effd640 1 -- 172.21.15.31:0/705465836 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c016070 con 0x55db44a84370 2024-09-16T00:44:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.747+0000 7f8b4effd640 1 -- 172.21.15.31:0/705465836 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5c004d20 con 0x55db44a84370 2024-09-16T00:44:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.747+0000 7f8b4effd640 1 -- 172.21.15.31:0/705465836 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c012870 con 0x55db44a84370 2024-09-16T00:44:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 -- 172.21.15.31:0/705465836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 msgr2=0x55db44a86760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 --2- 172.21.15.31:0/705465836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a86760 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009e60 tx=0x7f8b5c010740 comp rx=0 tx=0).stop 2024-09-16T00:44:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 -- 172.21.15.31:0/705465836 shutdown_connections 2024-09-16T00:44:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 --2- 172.21.15.31:0/705465836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a86760 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 --2- 172.21.15.31:0/705465836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a795d0 0x55db44a79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 --2- 172.21.15.31:0/705465836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 0x55db44a79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 -- 172.21.15.31:0/705465836 >> 172.21.15.31:0/705465836 conn(0x55db44a42310 msgr2=0x55db44a75c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 -- 172.21.15.31:0/705465836 shutdown_connections 2024-09-16T00:44:03.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 -- 172.21.15.31:0/705465836 wait complete. 2024-09-16T00:44:03.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 -- start start 2024-09-16T00:44:03.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.748+0000 7f8b6d148740 1 Processor -- start 2024-09-16T00:44:03.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a83d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a80fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a84370 0x55db44a81530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b6d148740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55db44a87070 con 0x55db44a795d0 2024-09-16T00:44:03.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b6d148740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55db4499a060 con 0x55db44a7ab40 2024-09-16T00:44:03.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b6d148740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55db44a871f0 con 0x55db44a84370 2024-09-16T00:44:03.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b6d148740 10 client.0 _ll_register_callbacks cb 0x55db446ed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:03.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b68b0b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a83d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b63fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a80fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b68b0b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a83d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50898/0 (socket says 172.21.15.31:50898) 2024-09-16T00:44:03.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b68b0b640 1 -- 172.21.15.31:0/2092268931 learned_addr learned my addr 172.21.15.31:0/2092268931 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:03.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b6930c640 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a84370 0x55db44a81530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b63fff640 1 -- 172.21.15.31:0/2092268931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a84370 msgr2=0x55db44a81530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:03.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b63fff640 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a84370 0x55db44a81530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b63fff640 1 -- 172.21.15.31:0/2092268931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 msgr2=0x55db44a83d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:03.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b63fff640 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a83d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b63fff640 1 -- 172.21.15.31:0/2092268931 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db447503c0 con 0x55db44a7ab40 2024-09-16T00:44:03.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a83d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:03.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b63fff640 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a80fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b50011fd0 tx=0x7f8b50004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18a476874858ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.749+0000 7f8b43fff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:44:03.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b43fff640 1 -- 172.21.15.31:0/2092268931 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5001cb60 con 0x55db44a7ab40 2024-09-16T00:44:03.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b6d148740 1 -- 172.21.15.31:0/2092268931 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b5c004d20 con 0x55db44a7ab40 2024-09-16T00:44:03.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b43fff640 1 -- 172.21.15.31:0/2092268931 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f8b5001cd00 con 0x55db44a7ab40 2024-09-16T00:44:03.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b43fff640 1 -- 172.21.15.31:0/2092268931 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8b50033050 con 0x55db44a7ab40 2024-09-16T00:44:03.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b6d148740 10 client.25430 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:03.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b6d148740 1 -- 172.21.15.31:0/2092268931 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55db44a92b30 con 0x55db44a7ab40 2024-09-16T00:44:03.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b43fff640 1 -- 172.21.15.31:0/2092268931 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5c004d20 con 0x55db44a7ab40 2024-09-16T00:44:03.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b43fff640 1 -- 172.21.15.31:0/2092268931 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f8b5001a040 con 0x55db44a7ab40 2024-09-16T00:44:03.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b6d148740 10 client.25430 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:03.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b6d148740 10 client.25430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:03.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b6d148740 20 client.25430 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:03.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b6d148740 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55db44a94040 0x55db44a964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b6d148740 4 client.25430 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:03.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.750+0000 7f8b6d148740 1 -- 172.21.15.31:0/2092268931 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db449ec860 con 0x55db44a94040 2024-09-16T00:44:03.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.751+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55db44a94040 0x55db44a964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.751+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55db44a94040 0x55db44a964a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad78de2afeba0cf server_cookie=dfc64bb62813bc48 in_seq=0 out_seq=0 2024-09-16T00:44:03.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.751+0000 7f8b43fff640 10 client.25430 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b43fff640 1 -- 172.21.15.31:0/2092268931 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db449ec860 con 0x55db44a94040 2024-09-16T00:44:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b43fff640 10 client.25430 handle_command_reply: tid=0 2024-09-16T00:44:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b6d148740 1 client.25430 shutdown 2024-09-16T00:44:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b6d148740 1 -- 172.21.15.31:0/2092268931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 msgr2=0x55db44a80fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b6d148740 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a80fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b50011fd0 tx=0x7f8b50004730 comp rx=0 tx=0).stop 2024-09-16T00:44:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b6d148740 1 -- 172.21.15.31:0/2092268931 shutdown_connections 2024-09-16T00:44:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b6d148740 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55db44a94040 0x55db44a964a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b6d148740 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a84370 0x55db44a81530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b6d148740 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a80fb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b6d148740 1 --2- 172.21.15.31:0/2092268931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a83d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b43fff640 0 client.25430 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.752+0000 7f8b6d148740 1 -- 172.21.15.31:0/2092268931 >> 172.21.15.31:0/2092268931 conn(0x55db44a42310 msgr2=0x55db44a7b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.753+0000 7f8b6d148740 1 -- 172.21.15.31:0/2092268931 shutdown_connections 2024-09-16T00:44:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.753+0000 7f8b6d148740 1 -- 172.21.15.31:0/2092268931 wait complete. 2024-09-16T00:44:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.753+0000 7f8b6d148740 20 client.25430 trim_cache size 0 max 16384 2024-09-16T00:44:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.768+0000 7f8b6d148740 1 Processor -- start 2024-09-16T00:44:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.768+0000 7f8b6d148740 1 -- start start 2024-09-16T00:44:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a795d0 0x55db44a90340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a7c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b6d148740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641aeef0 con 0x55db44a84370 2024-09-16T00:44:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b6d148740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641b1770 con 0x55db44a7ab40 2024-09-16T00:44:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b6d148740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641af560 con 0x55db44a795d0 2024-09-16T00:44:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b63fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b63fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a7c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:57374/0 (socket says 172.21.15.31:57374) 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b63fff640 1 -- 172.21.15.31:0/1789801314 learned_addr learned my addr 172.21.15.31:0/1789801314 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b6930c640 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a7c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a795d0 0x55db44a90340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b63fff640 1 -- 172.21.15.31:0/1789801314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a795d0 msgr2=0x55db44a90340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b63fff640 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a795d0 0x55db44a90340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b63fff640 1 -- 172.21.15.31:0/1789801314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 msgr2=0x55db44a7c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b63fff640 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a7c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b63fff640 1 -- 172.21.15.31:0/1789801314 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db44a7cba0 con 0x55db44a7ab40 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.769+0000 7f8b6930c640 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.770+0000 7f8b63fff640 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a7c120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b50011d10 tx=0x7f8b5002e3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c0aa9496fca9f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.770+0000 7f8b4e7fc640 1 -- 172.21.15.31:0/1789801314 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5003c070 con 0x55db44a7ab40 2024-09-16T00:44:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.770+0000 7f8b4e7fc640 1 -- 172.21.15.31:0/1789801314 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5002a070 con 0x55db44a7ab40 2024-09-16T00:44:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.770+0000 7f8b4e7fc640 1 -- 172.21.15.31:0/1789801314 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5001cd60 con 0x55db44a7ab40 2024-09-16T00:44:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.771+0000 7f8b6d148740 1 -- 172.21.15.31:0/1789801314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 msgr2=0x55db44a7c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.771+0000 7f8b6d148740 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a7c120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b50011d10 tx=0x7f8b5002e3b0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.771+0000 7f8b6d148740 1 -- 172.21.15.31:0/1789801314 shutdown_connections 2024-09-16T00:44:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.771+0000 7f8b6d148740 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a84370 0x55db44a7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.771+0000 7f8b6d148740 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a7ab40 0x55db44a7c120 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.771+0000 7f8b6d148740 1 --2- 172.21.15.31:0/1789801314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a795d0 0x55db44a90340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.771+0000 7f8b6d148740 1 -- 172.21.15.31:0/1789801314 >> 172.21.15.31:0/1789801314 conn(0x55db44a42310 msgr2=0x55db44a800a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.771+0000 7f8b6d148740 1 -- 172.21.15.31:0/1789801314 shutdown_connections 2024-09-16T00:44:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.771+0000 7f8b6d148740 1 -- 172.21.15.31:0/1789801314 wait complete. 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6d148740 1 -- start start 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6d148740 1 Processor -- start 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a8cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 0x55db44a88090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6d148740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a84370 0x55db44a88820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6d148740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641afc30 con 0x55db44a795d0 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6d148740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641b10a0 con 0x55db44a84370 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6d148740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641b0300 con 0x55db44a7ab40 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6d148740 10 client.0 _ll_register_callbacks cb 0x7f8b50009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b68b0b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a8cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6930c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a84370 0x55db44a88820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6930c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a84370 0x55db44a88820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:57382/0 (socket says 172.21.15.31:57382) 2024-09-16T00:44:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.772+0000 7f8b6930c640 1 -- 172.21.15.31:0/4191006173 learned_addr learned my addr 172.21.15.31:0/4191006173 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b63fff640 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 0x55db44a88090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b68b0b640 1 -- 172.21.15.31:0/4191006173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 msgr2=0x55db44a88090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 0x55db44a88090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b68b0b640 1 -- 172.21.15.31:0/4191006173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a84370 msgr2=0x55db44a88820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a84370 0x55db44a88820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b68b0b640 1 -- 172.21.15.31:0/4191006173 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db44afdd40 con 0x55db44a795d0 2024-09-16T00:44:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b6930c640 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a84370 0x55db44a88820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b63fff640 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 0x55db44a88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b68b0b640 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a8cd60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b580edb80 tx=0x7f8b5803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5431d5a5d6ee4d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b43fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:44:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b43fff640 1 -- 172.21.15.31:0/4191006173 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58049040 con 0x55db44a795d0 2024-09-16T00:44:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b43fff640 1 -- 172.21.15.31:0/4191006173 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f8b580f00d0 con 0x55db44a795d0 2024-09-16T00:44:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b6d148740 1 -- 172.21.15.31:0/4191006173 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b5002a070 con 0x55db44a795d0 2024-09-16T00:44:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b43fff640 1 -- 172.21.15.31:0/4191006173 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8b580c52c0 con 0x55db44a795d0 2024-09-16T00:44:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b6d148740 10 client.15753 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.773+0000 7f8b6d148740 1 -- 172.21.15.31:0/4191006173 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55db44afef00 con 0x55db44a795d0 2024-09-16T00:44:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b43fff640 1 -- 172.21.15.31:0/4191006173 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5002a070 con 0x55db44a795d0 2024-09-16T00:44:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b43fff640 1 -- 172.21.15.31:0/4191006173 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f8b580c4e30 con 0x55db44a795d0 2024-09-16T00:44:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b6d148740 10 client.15753 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b6d148740 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b6d148740 20 client.15753 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b6d148740 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55db44aff5d0 0x55db44b019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b6d148740 4 client.15753 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:44:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b6d148740 1 -- 172.21.15.31:0/4191006173 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db449e6d70 con 0x55db44aff5d0 2024-09-16T00:44:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b63fff640 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55db44aff5d0 0x55db44b019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b43fff640 10 client.15753 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.774+0000 7f8b63fff640 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55db44aff5d0 0x55db44b019c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11bdfbcf58c64c55 server_cookie=e6f140a635d56102 in_seq=0 out_seq=0 2024-09-16T00:44:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b43fff640 1 -- 172.21.15.31:0/4191006173 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db449e6d70 con 0x55db44aff5d0 2024-09-16T00:44:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b43fff640 10 client.15753 handle_command_reply: tid=0 2024-09-16T00:44:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b6d148740 1 client.15753 shutdown 2024-09-16T00:44:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b6d148740 1 -- 172.21.15.31:0/4191006173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 msgr2=0x55db44a8cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b6d148740 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a8cd60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b580edb80 tx=0x7f8b5803b800 comp rx=0 tx=0).stop 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b6d148740 1 -- 172.21.15.31:0/4191006173 shutdown_connections 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b6d148740 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db44a84370 0x55db44a88820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b6d148740 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55db44aff5d0 0x55db44b019c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b6d148740 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db44a7ab40 0x55db44a88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b6d148740 1 --2- 172.21.15.31:0/4191006173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db44a795d0 0x55db44a8cd60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b43fff640 0 client.15753 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.776+0000 7f8b6d148740 1 -- 172.21.15.31:0/4191006173 >> 172.21.15.31:0/4191006173 conn(0x55db44a42310 msgr2=0x55db44a82810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6d148740 1 -- 172.21.15.31:0/4191006173 shutdown_connections 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6d148740 1 -- 172.21.15.31:0/4191006173 wait complete. 2024-09-16T00:44:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6d148740 20 client.15753 trim_cache size 0 max 16384 2024-09-16T00:44:03.784 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:44:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8b540a3240 msgr2=0x7f8b540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8b540a3240 0x7f8b540a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b500095f0 tx=0x7f8b50005f90 comp rx=0 tx=0).stop 2024-09-16T00:44:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 msgr2=0x7f8b640fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 0x7f8b640fcc80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8b5802aa70 tx=0x7f8b58003c10 comp rx=0 tx=0).stop 2024-09-16T00:44:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/4123750840 shutdown_connections 2024-09-16T00:44:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f8b540a3240 0x7f8b540a5700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b64102ef0 0x7f8b640fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b64102570 0x7f8b640fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 --2- 172.21.15.31:0/4123750840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b64100e20 0x7f8b640fcc80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/4123750840 >> 172.21.15.31:0/4123750840 conn(0x7f8b640fc7d0 msgr2=0x7f8b6406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/4123750840 shutdown_connections 2024-09-16T00:44:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:03.777+0000 7f8b6ad6f640 1 -- 172.21.15.31:0/4123750840 wait complete. 2024-09-16T00:44:03.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:03 smithi102 ceph-mon[35364]: pgmap v443: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 500 KiB/s wr, 170 op/s 2024-09-16T00:44:05.656 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:44:05.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:44:05.705 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:05 smithi031 ceph-mon[30956]: pgmap v444: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 154 KiB/s wr, 134 op/s 2024-09-16T00:44:05.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:05 smithi081 ceph-mon[35539]: pgmap v444: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 154 KiB/s wr, 134 op/s 2024-09-16T00:44:05.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:05 smithi102 ceph-mon[35364]: pgmap v444: 129 pgs: 129 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 154 KiB/s wr, 134 op/s 2024-09-16T00:44:05.978 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:06.187 INFO:tasks.workunit.client.0.smithi031.stdout: 50 66048 122 42553 84 38793 219 2024-09-16T00:44:06.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.453+0000 7f535381f640 1 -- 172.21.15.31:0/228320210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102570 msgr2=0x7f534c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:06.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.453+0000 7f535381f640 1 --2- 172.21.15.31:0/228320210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102570 0x7f534c1029b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f533c009900 tx=0x7f533c02a480 comp rx=0 tx=0).stop 2024-09-16T00:44:06.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.453+0000 7f535381f640 1 -- 172.21.15.31:0/228320210 shutdown_connections 2024-09-16T00:44:06.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.453+0000 7f535381f640 1 --2- 172.21.15.31:0/228320210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f534c102ef0 0x7f534c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.453+0000 7f535381f640 1 --2- 172.21.15.31:0/228320210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102570 0x7f534c1029b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.453+0000 7f535381f640 1 --2- 172.21.15.31:0/228320210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c100e20 0x7f534c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.453+0000 7f535381f640 1 -- 172.21.15.31:0/228320210 >> 172.21.15.31:0/228320210 conn(0x7f534c0fc7d0 msgr2=0x7f534c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.453+0000 7f535381f640 1 -- 172.21.15.31:0/228320210 shutdown_connections 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.453+0000 7f535381f640 1 -- 172.21.15.31:0/228320210 wait complete. 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f535381f640 1 Processor -- start 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f535381f640 1 -- start start 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f535381f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c100e20 0x7f534c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f535381f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f534c102570 0x7f534c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f535381f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102ef0 0x7f534c1a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f535381f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f534c110fa0 con 0x7f534c102ef0 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f535381f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f534c110e20 con 0x7f534c102570 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f535381f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f534c111120 con 0x7f534c100e20 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f5351dbc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102ef0 0x7f534c1a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f5350dba640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f534c102570 0x7f534c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f5351dbc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102ef0 0x7f534c1a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50936/0 (socket says 172.21.15.31:50936) 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.454+0000 7f5351dbc640 1 -- 172.21.15.31:0/2806880786 learned_addr learned my addr 172.21.15.31:0/2806880786 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f53515bb640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c100e20 0x7f534c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f5351dbc640 1 -- 172.21.15.31:0/2806880786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c100e20 msgr2=0x7f534c1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f5351dbc640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c100e20 0x7f534c1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f5351dbc640 1 -- 172.21.15.31:0/2806880786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f534c102570 msgr2=0x7f534c1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f5351dbc640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f534c102570 0x7f534c1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f5351dbc640 1 -- 172.21.15.31:0/2806880786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c1a4970 con 0x7f534c102ef0 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f53515bb640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c100e20 0x7f534c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f5350dba640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f534c102570 0x7f534c1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f5351dbc640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102ef0 0x7f534c1a4430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f534400eb10 tx=0x7f534400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f53427fc640 1 -- 172.21.15.31:0/2806880786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344004b20 con 0x7f534c102ef0 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f53427fc640 1 -- 172.21.15.31:0/2806880786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5344004cc0 con 0x7f534c102ef0 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f535381f640 1 -- 172.21.15.31:0/2806880786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c1a4b40 con 0x7f534c102ef0 2024-09-16T00:44:06.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f53427fc640 1 -- 172.21.15.31:0/2806880786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344010760 con 0x7f534c102ef0 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.455+0000 7f535381f640 1 -- 172.21.15.31:0/2806880786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c1a5000 con 0x7f534c102ef0 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.458+0000 7f53427fc640 1 -- 172.21.15.31:0/2806880786 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5344005020 con 0x7f534c102ef0 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.458+0000 7f5355bf8740 1 Processor -- start 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.458+0000 7f5355bf8740 1 -- start start 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.458+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 0x564a83903f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.458+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83904480 0x564a8390e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.458+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a8390ed40 0x564a83911130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.458+0000 7f5355bf8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564a83869280 con 0x564a8390ed40 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.458+0000 7f5355bf8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564a83810b10 con 0x564a83904480 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.458+0000 7f5355bf8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564a83865240 con 0x564a83903b60 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53515bb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83904480 0x564a8390e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f5350dba640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 0x564a83903f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53515bb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83904480 0x564a8390e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:57406/0 (socket says 172.21.15.31:57406) 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53515bb640 1 -- 172.21.15.31:0/4128171262 learned_addr learned my addr 172.21.15.31:0/4128171262 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53427fc640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f53380a3240 0x7f53380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f5350dba640 1 --2- 172.21.15.31:0/4128171262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a8390ed40 0x564a83911130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f5351dbc640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f53380a3240 0x7f53380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53515bb640 1 -- 172.21.15.31:0/4128171262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 msgr2=0x564a83903f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53515bb640 1 --2- 172.21.15.31:0/4128171262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 0x564a83903f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53515bb640 1 -- 172.21.15.31:0/4128171262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a8390ed40 msgr2=0x564a83911130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53515bb640 1 --2- 172.21.15.31:0/4128171262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a8390ed40 0x564a83911130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53515bb640 1 -- 172.21.15.31:0/4128171262 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a8381d1f0 con 0x564a83904480 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53427fc640 1 -- 172.21.15.31:0/2806880786 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f53440d0380 con 0x7f534c102ef0 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.459+0000 7f53515bb640 1 --2- 172.21.15.31:0/4128171262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83904480 0x564a8390e800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5348007850 tx=0x7f5348007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de1145510796a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:06.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5351dbc640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f53380a3240 0x7f53380a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f53440989f0 tx=0x7f53440673c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f53377fe640 1 -- 172.21.15.31:0/4128171262 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348016be0 con 0x564a83904480 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f53377fe640 1 -- 172.21.15.31:0/4128171262 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5348016d80 con 0x564a83904480 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f53377fe640 1 -- 172.21.15.31:0/4128171262 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534801e9c0 con 0x564a83904480 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5355bf8740 1 -- 172.21.15.31:0/4128171262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83904480 msgr2=0x564a8390e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5355bf8740 1 --2- 172.21.15.31:0/4128171262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83904480 0x564a8390e800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5348007850 tx=0x7f5348007d10 comp rx=0 tx=0).stop 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5355bf8740 1 -- 172.21.15.31:0/4128171262 shutdown_connections 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5355bf8740 1 --2- 172.21.15.31:0/4128171262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a8390ed40 0x564a83911130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5355bf8740 1 --2- 172.21.15.31:0/4128171262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83904480 0x564a8390e800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5355bf8740 1 --2- 172.21.15.31:0/4128171262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 0x564a83903f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5355bf8740 1 -- 172.21.15.31:0/4128171262 >> 172.21.15.31:0/4128171262 conn(0x564a839000f0 msgr2=0x564a839004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5355bf8740 1 -- 172.21.15.31:0/4128171262 shutdown_connections 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.460+0000 7f5355bf8740 1 -- 172.21.15.31:0/4128171262 wait complete. 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5355bf8740 1 -- start start 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5355bf8740 1 Processor -- start 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83903b60 0x564a8390e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a8390bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a8390ed40 0x564a8390c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5355bf8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564a83911910 con 0x564a83904480 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5355bf8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564a83823f10 con 0x564a83903b60 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5355bf8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564a83911a90 con 0x564a8390ed40 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5355bf8740 10 client.0 _ll_register_callbacks cb 0x564a837eca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f53515bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a8390bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f53515bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a8390bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50960/0 (socket says 172.21.15.31:50960) 2024-09-16T00:44:06.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f53515bb640 1 -- 172.21.15.31:0/1025134905 learned_addr learned my addr 172.21.15.31:0/1025134905 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.461+0000 7f5350dba640 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83903b60 0x564a8390e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f5350dba640 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a8390ed40 0x564a8390c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f53515bb640 1 -- 172.21.15.31:0/1025134905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a8390ed40 msgr2=0x564a8390c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f53515bb640 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a8390ed40 0x564a8390c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f53515bb640 1 -- 172.21.15.31:0/1025134905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83903b60 msgr2=0x564a8390e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f53515bb640 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83903b60 0x564a8390e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f53515bb640 1 -- 172.21.15.31:0/1025134905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a835da6a0 con 0x564a83904480 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f53515bb640 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a8390bb20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f534801e540 tx=0x7f5348016c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cacee2b280c15c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f5334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f5334ff9640 1 -- 172.21.15.31:0/1025134905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348024040 con 0x564a83904480 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f5355bf8740 1 -- 172.21.15.31:0/1025134905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5348016d80 con 0x564a83904480 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f5334ff9640 1 -- 172.21.15.31:0/1025134905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f534801e740 con 0x564a83904480 2024-09-16T00:44:06.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f5334ff9640 1 -- 172.21.15.31:0/1025134905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5348037050 con 0x564a83904480 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f5355bf8740 10 client.15765 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f5355bf8740 1 -- 172.21.15.31:0/1025134905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564a8391d420 con 0x564a83904480 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.462+0000 7f5334ff9640 1 -- 172.21.15.31:0/1025134905 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5348016d80 con 0x564a83904480 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5334ff9640 1 -- 172.21.15.31:0/1025134905 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f5348027030 con 0x564a83904480 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5355bf8740 10 client.15765 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5355bf8740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5355bf8740 20 client.15765 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5355bf8740 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564a8391e7d0 0x564a83920bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5355bf8740 4 client.15765 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5355bf8740 1 -- 172.21.15.31:0/1025134905 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a838767f0 con 0x564a8391e7d0 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5350dba640 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564a8391e7d0 0x564a83920bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5350dba640 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564a8391e7d0 0x564a83920bc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1717f343ef49e0 server_cookie=e59acd5e15da34d3 in_seq=0 out_seq=0 2024-09-16T00:44:06.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.463+0000 7f5334ff9640 10 client.15765 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5334ff9640 1 -- 172.21.15.31:0/1025134905 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a838767f0 con 0x564a8391e7d0 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5334ff9640 10 client.15765 handle_command_reply: tid=0 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 client.15765 shutdown 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 -- 172.21.15.31:0/1025134905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 msgr2=0x564a8390bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a8390bb20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f534801e540 tx=0x7f5348016c90 comp rx=0 tx=0).stop 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 -- 172.21.15.31:0/1025134905 shutdown_connections 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564a8391e7d0 0x564a83920bc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a8390ed40 0x564a8390c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a8390bb20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 --2- 172.21.15.31:0/1025134905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a83903b60 0x564a8390e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 -- 172.21.15.31:0/1025134905 >> 172.21.15.31:0/1025134905 conn(0x564a839000f0 msgr2=0x564a838ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5334ff9640 0 client.15765 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 -- 172.21.15.31:0/1025134905 shutdown_connections 2024-09-16T00:44:06.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 1 -- 172.21.15.31:0/1025134905 wait complete. 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.465+0000 7f5355bf8740 20 client.15765 trim_cache size 0 max 16384 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5355bf8740 1 Processor -- start 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5355bf8740 1 -- start start 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 0x564a8390a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a839261f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 0x564a83926730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5355bf8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1aab60 con 0x564a83904480 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5355bf8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1ad3e0 con 0x564a8390ed40 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5355bf8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1ab1d0 con 0x564a83903b60 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f53515bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a839261f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f53515bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a839261f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50976/0 (socket says 172.21.15.31:50976) 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f53515bb640 1 -- 172.21.15.31:0/3596367534 learned_addr learned my addr 172.21.15.31:0/3596367534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5350dba640 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 0x564a8390a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.483+0000 7f5350dba640 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 0x564a83926730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f53515bb640 1 -- 172.21.15.31:0/3596367534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 msgr2=0x564a8390a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f53515bb640 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 0x564a8390a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f53515bb640 1 -- 172.21.15.31:0/3596367534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 msgr2=0x564a83926730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f53515bb640 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 0x564a83926730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f53515bb640 1 -- 172.21.15.31:0/3596367534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a838fff20 con 0x564a83904480 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f5350dba640 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 0x564a8390a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f53515bb640 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a839261f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5348000c00 tx=0x7f5348007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8829d0fec240533 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f5336ffd640 1 -- 172.21.15.31:0/3596367534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534801f430 con 0x564a83904480 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f5336ffd640 1 -- 172.21.15.31:0/3596367534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f534800ddc0 con 0x564a83904480 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.484+0000 7f5336ffd640 1 -- 172.21.15.31:0/3596367534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534801fba0 con 0x564a83904480 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.485+0000 7f5355bf8740 1 -- 172.21.15.31:0/3596367534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 msgr2=0x564a839261f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.485+0000 7f5355bf8740 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a839261f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5348000c00 tx=0x7f5348007750 comp rx=0 tx=0).stop 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.485+0000 7f5355bf8740 1 -- 172.21.15.31:0/3596367534 shutdown_connections 2024-09-16T00:44:06.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.485+0000 7f5355bf8740 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 0x564a83926730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.485+0000 7f5355bf8740 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83904480 0x564a839261f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.485+0000 7f5355bf8740 1 --2- 172.21.15.31:0/3596367534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83903b60 0x564a8390a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.485+0000 7f5355bf8740 1 -- 172.21.15.31:0/3596367534 >> 172.21.15.31:0/3596367534 conn(0x564a839000f0 msgr2=0x564a8390a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.485+0000 7f5355bf8740 1 -- 172.21.15.31:0/3596367534 shutdown_connections 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.485+0000 7f5355bf8740 1 -- 172.21.15.31:0/3596367534 wait complete. 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5355bf8740 1 -- start start 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5355bf8740 1 Processor -- start 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83903b60 0x564a83917480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83904480 0x564a839137c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5355bf8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 0x564a83913d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5355bf8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1ab8a0 con 0x564a83903b60 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5355bf8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1acd10 con 0x564a8390ed40 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5355bf8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1abf70 con 0x564a83904480 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5355bf8740 10 client.0 _ll_register_callbacks cb 0x7f533c0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5350dba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83903b60 0x564a83917480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5350dba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83903b60 0x564a83917480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50992/0 (socket says 172.21.15.31:50992) 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5350dba640 1 -- 172.21.15.31:0/509825130 learned_addr learned my addr 172.21.15.31:0/509825130 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f53515bb640 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83904480 0x564a839137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.486+0000 7f5350dba640 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 0x564a83913d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5350dba640 1 -- 172.21.15.31:0/509825130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83904480 msgr2=0x564a839137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5350dba640 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83904480 0x564a839137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5350dba640 1 -- 172.21.15.31:0/509825130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 msgr2=0x564a83913d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5350dba640 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 0x564a83913d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5350dba640 1 -- 172.21.15.31:0/509825130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a839011d0 con 0x564a83903b60 2024-09-16T00:44:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5350dba640 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83903b60 0x564a83917480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f533c0314e0 tx=0x7f533c03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a5810b5ce0991e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:44:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5334ff9640 1 -- 172.21.15.31:0/509825130 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c035040 con 0x564a83903b60 2024-09-16T00:44:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5355bf8740 1 -- 172.21.15.31:0/509825130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f534800ddc0 con 0x564a83903b60 2024-09-16T00:44:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5334ff9640 1 -- 172.21.15.31:0/509825130 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f533c005500 con 0x564a83903b60 2024-09-16T00:44:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5334ff9640 1 -- 172.21.15.31:0/509825130 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f533c0047a0 con 0x564a83903b60 2024-09-16T00:44:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5355bf8740 10 client.15771 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5355bf8740 1 -- 172.21.15.31:0/509825130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564a8398a100 con 0x564a83903b60 2024-09-16T00:44:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.487+0000 7f5334ff9640 1 -- 172.21.15.31:0/509825130 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f534800ddc0 con 0x564a83903b60 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f5334ff9640 1 -- 172.21.15.31:0/509825130 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f533c031a90 con 0x564a83903b60 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f5355bf8740 10 client.15771 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f5355bf8740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f5355bf8740 20 client.15771 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f5355bf8740 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564a8398a7d0 0x564a8398cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f5355bf8740 4 client.15771 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f5355bf8740 1 -- 172.21.15.31:0/509825130 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564a83870d00 con 0x564a8398a7d0 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f53515bb640 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564a8398a7d0 0x564a8398cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f5334ff9640 10 client.15771 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:06.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.488+0000 7f53515bb640 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564a8398a7d0 0x564a8398cbc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b4c4eecf0a57d4 server_cookie=31d506c0ea096d34 in_seq=0 out_seq=0 2024-09-16T00:44:06.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5334ff9640 1 -- 172.21.15.31:0/509825130 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564a83870d00 con 0x564a8398a7d0 2024-09-16T00:44:06.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5334ff9640 10 client.15771 handle_command_reply: tid=0 2024-09-16T00:44:06.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 client.15771 shutdown 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 -- 172.21.15.31:0/509825130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83903b60 msgr2=0x564a83917480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83903b60 0x564a83917480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f533c0314e0 tx=0x7f533c03f040 comp rx=0 tx=0).stop 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 -- 172.21.15.31:0/509825130 shutdown_connections 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a8390ed40 0x564a83913d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564a8398a7d0 0x564a8398cbc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a83904480 0x564a839137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 --2- 172.21.15.31:0/509825130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a83903b60 0x564a83917480 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 -- 172.21.15.31:0/509825130 >> 172.21.15.31:0/509825130 conn(0x564a839000f0 msgr2=0x564a8390aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5334ff9640 0 client.15771 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.503+0000 7f5355bf8740 1 -- 172.21.15.31:0/509825130 shutdown_connections 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.504+0000 7f5355bf8740 1 -- 172.21.15.31:0/509825130 wait complete. 2024-09-16T00:44:06.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.504+0000 7f5355bf8740 20 client.15771 trim_cache size 0 max 16384 2024-09-16T00:44:06.514 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:44:06.514 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:44:06.514 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "e4ab83eb-50aa-440e-9051-fc9a63771276", 2024-09-16T00:44:06.514 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:44:06.514 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:44:06.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.504+0000 7f535381f640 1 -- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f53380a3240 msgr2=0x7f53380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:06.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.504+0000 7f535381f640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f53380a3240 0x7f53380a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f53440989f0 tx=0x7f53440673c0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.504+0000 7f535381f640 1 -- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102ef0 msgr2=0x7f534c1a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.504+0000 7f535381f640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102ef0 0x7f534c1a4430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f534400eb10 tx=0x7f534400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.505+0000 7f535381f640 1 -- 172.21.15.31:0/2806880786 shutdown_connections 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.505+0000 7f535381f640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c102ef0 0x7f534c1a4430 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.505+0000 7f535381f640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f534c102570 0x7f534c1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.505+0000 7f535381f640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f53380a3240 0x7f53380a5700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.505+0000 7f535381f640 1 --2- 172.21.15.31:0/2806880786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c100e20 0x7f534c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.505+0000 7f535381f640 1 -- 172.21.15.31:0/2806880786 >> 172.21.15.31:0/2806880786 conn(0x7f534c0fc7d0 msgr2=0x7f534c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.505+0000 7f535381f640 1 -- 172.21.15.31:0/2806880786 shutdown_connections 2024-09-16T00:44:06.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:06.505+0000 7f535381f640 1 -- 172.21.15.31:0/2806880786 wait complete. 2024-09-16T00:44:07.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:44:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:07 smithi031 ceph-mon[30956]: pgmap v445: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 202 KiB/s wr, 80 op/s 2024-09-16T00:44:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:07 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:44:07.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:07 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:07.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:07 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:44:07.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:07 smithi081 ceph-mon[35539]: pgmap v445: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 202 KiB/s wr, 80 op/s 2024-09-16T00:44:07.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:07 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:44:07.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:07 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:07.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:07 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:44:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:07 smithi102 ceph-mon[35364]: pgmap v445: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 202 KiB/s wr, 80 op/s 2024-09-16T00:44:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:07 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:44:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:07 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:07.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:07 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:44:08.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.307+0000 7f05d15a8640 1 -- 172.21.15.31:0/4203288322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 msgr2=0x7f05cc105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.307+0000 7f05d15a8640 1 --2- 172.21.15.31:0/4203288322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 0x7f05cc105920 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009900 tx=0x7f05c002a470 comp rx=0 tx=0).stop 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.307+0000 7f05d15a8640 1 -- 172.21.15.31:0/4203288322 shutdown_connections 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.307+0000 7f05d15a8640 1 --2- 172.21.15.31:0/4203288322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05cc105f40 0x7f05cc1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.307+0000 7f05d15a8640 1 --2- 172.21.15.31:0/4203288322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 0x7f05cc105920 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.307+0000 7f05d15a8640 1 --2- 172.21.15.31:0/4203288322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05cc100b30 0x7f05cc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.307+0000 7f05d15a8640 1 -- 172.21.15.31:0/4203288322 >> 172.21.15.31:0/4203288322 conn(0x7f05cc0fc7a0 msgr2=0x7f05cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.308+0000 7f05d15a8640 1 -- 172.21.15.31:0/4203288322 shutdown_connections 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.308+0000 7f05d15a8640 1 -- 172.21.15.31:0/4203288322 wait complete. 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.308+0000 7f05d15a8640 1 Processor -- start 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.308+0000 7f05d15a8640 1 -- start start 2024-09-16T00:44:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05d15a8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05cc100b30 0x7f05cc1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05d15a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 0x7f05cc1a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05d15a8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05cc105f40 0x7f05cc19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05d15a8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc118560 con 0x7f05cc101450 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05d15a8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1183e0 con 0x7f05cc105f40 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05d15a8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1186e0 con 0x7f05cc100b30 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05ca7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 0x7f05cc1a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05ca7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 0x7f05cc1a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54484/0 (socket says 172.21.15.31:54484) 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05ca7fc640 1 -- 172.21.15.31:0/1099022072 learned_addr learned my addr 172.21.15.31:0/1099022072 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05cb7fe640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05cc105f40 0x7f05cc19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05caffd640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05cc100b30 0x7f05cc1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05ca7fc640 1 -- 172.21.15.31:0/1099022072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05cc100b30 msgr2=0x7f05cc1a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05cc100b30 0x7f05cc1a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05ca7fc640 1 -- 172.21.15.31:0/1099022072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05cc105f40 msgr2=0x7f05cc19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05cc105f40 0x7f05cc19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05ca7fc640 1 -- 172.21.15.31:0/1099022072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05cc19f040 con 0x7f05cc101450 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05cb7fe640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05cc105f40 0x7f05cc19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05caffd640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05cc100b30 0x7f05cc1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.309+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 0x7f05cc1a4c90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f05c0006010 tx=0x7f05c0031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.310+0000 7f05b3fff640 1 -- 172.21.15.31:0/1099022072 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c003f030 con 0x7f05cc101450 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.310+0000 7f05b3fff640 1 -- 172.21.15.31:0/1099022072 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c00043d0 con 0x7f05cc101450 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.310+0000 7f05d15a8640 1 -- 172.21.15.31:0/1099022072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05cc19f2d0 con 0x7f05cc101450 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.310+0000 7f05b3fff640 1 -- 172.21.15.31:0/1099022072 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0035070 con 0x7f05cc101450 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.310+0000 7f05d15a8640 1 -- 172.21.15.31:0/1099022072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05cc1ab620 con 0x7f05cc101450 2024-09-16T00:44:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05b3fff640 1 -- 172.21.15.31:0/1099022072 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f05c0042030 con 0x7f05cc101450 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05d3981740 1 Processor -- start 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05d3981740 1 -- start start 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05d3981740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05d3981740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 0x5562f65db140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05d3981740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 0x5562f65dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05d3981740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5562f65362e0 con 0x5562f65db680 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05d3981740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5562f64ddce0 con 0x5562f65d04a0 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05d3981740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5562f65322a0 con 0x5562f65d0dc0 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05cb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05ca7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 0x5562f65dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05cb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:51856/0 (socket says 172.21.15.31:51856) 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05cb7fe640 1 -- 172.21.15.31:0/4123662559 learned_addr learned my addr 172.21.15.31:0/4123662559 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.313+0000 7f05caffd640 1 --2- 172.21.15.31:0/4123662559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 0x5562f65db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05cb7fe640 1 -- 172.21.15.31:0/4123662559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 msgr2=0x5562f65db140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05cb7fe640 1 --2- 172.21.15.31:0/4123662559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 0x5562f65db140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05cb7fe640 1 -- 172.21.15.31:0/4123662559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 msgr2=0x5562f65dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05cb7fe640 1 --2- 172.21.15.31:0/4123662559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 0x5562f65dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05cb7fe640 1 -- 172.21.15.31:0/4123662559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f64c2440 con 0x5562f65d04a0 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05b3fff640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05b40a3240 0x7f05b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05cb7fe640 1 --2- 172.21.15.31:0/4123662559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65d0880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05c4011790 tx=0x7f05c4011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af1c3bf810dbfe7f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05caffd640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05b40a3240 0x7f05b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05b3fff640 1 -- 172.21.15.31:0/1099022072 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f05c00eff60 con 0x7f05cc101450 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05b17fa640 1 -- 172.21.15.31:0/4123662559 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4003c30 con 0x5562f65d04a0 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05b17fa640 1 -- 172.21.15.31:0/4123662559 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c400fe80 con 0x5562f65d04a0 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.314+0000 7f05b17fa640 1 -- 172.21.15.31:0/4123662559 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c40193e0 con 0x5562f65d04a0 2024-09-16T00:44:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 -- 172.21.15.31:0/4123662559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 msgr2=0x5562f65d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05caffd640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05b40a3240 0x7f05b40a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5562f65cd890 tx=0x7f05bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 --2- 172.21.15.31:0/4123662559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65d0880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05c4011790 tx=0x7f05c4011c50 comp rx=0 tx=0).stop 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 -- 172.21.15.31:0/4123662559 shutdown_connections 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 --2- 172.21.15.31:0/4123662559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 0x5562f65dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 --2- 172.21.15.31:0/4123662559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 0x5562f65db140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 --2- 172.21.15.31:0/4123662559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65d0880 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 -- 172.21.15.31:0/4123662559 >> 172.21.15.31:0/4123662559 conn(0x5562f65cca30 msgr2=0x5562f65cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 -- 172.21.15.31:0/4123662559 shutdown_connections 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 -- 172.21.15.31:0/4123662559 wait complete. 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 -- start start 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.315+0000 7f05d3981740 1 Processor -- start 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05d3981740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05d3981740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 0x5562f65d8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05d3981740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 0x5562f65d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05d3981740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5562f65de250 con 0x5562f65d04a0 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05d3981740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5562f64f1050 con 0x5562f65d0dc0 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05d3981740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5562f65de3d0 con 0x5562f65db680 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05d3981740 10 client.0 _ll_register_callbacks cb 0x5562f6531160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05caffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05caffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65dafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54494/0 (socket says 172.21.15.31:54494) 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05caffd640 1 -- 172.21.15.31:0/3209132842 learned_addr learned my addr 172.21.15.31:0/3209132842 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 0x5562f65d8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05cb7fe640 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 0x5562f65d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05caffd640 1 -- 172.21.15.31:0/3209132842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 msgr2=0x5562f65d87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05caffd640 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 0x5562f65d87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05caffd640 1 -- 172.21.15.31:0/3209132842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 msgr2=0x5562f65d8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05caffd640 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 0x5562f65d8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05caffd640 1 -- 172.21.15.31:0/3209132842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f62a73c0 con 0x5562f65d04a0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05caffd640 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65dafe0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00fb50 tx=0x7f05bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a838497b6dfe1da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.316+0000 7f05aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.317+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209132842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc004020 con 0x5562f65d04a0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.317+0000 7f05d3981740 1 -- 172.21.15.31:0/3209132842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562f6543850 con 0x5562f65d04a0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.317+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209132842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f05bc018030 con 0x5562f65d04a0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.317+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209132842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f05bc033050 con 0x5562f65d04a0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.317+0000 7f05d3981740 10 client.15783 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.317+0000 7f05d3981740 1 -- 172.21.15.31:0/3209132842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5562f65e9c80 con 0x5562f65d04a0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.317+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209132842 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5562f6543850 con 0x5562f65d04a0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.317+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209132842 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f05bc01e5a0 con 0x5562f65d04a0 2024-09-16T00:44:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.317+0000 7f05d3981740 10 client.15783 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.318+0000 7f05d3981740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.318+0000 7f05d3981740 20 client.15783 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.318+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5562f65eb140 0x5562f65ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.318+0000 7f05d3981740 4 client.15783 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.318+0000 7f05d3981740 1 -- 172.21.15.31:0/3209132842 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562f653dd60 con 0x5562f65eb140 2024-09-16T00:44:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.318+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5562f65eb140 0x5562f65ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.318+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5562f65eb140 0x5562f65ed5c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0f052841fe0446 server_cookie=259f75ee7971cc55 in_seq=0 out_seq=0 2024-09-16T00:44:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.318+0000 7f05aeff5640 10 client.15783 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209132842 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562f653dd60 con 0x5562f65eb140 2024-09-16T00:44:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05aeff5640 10 client.15783 handle_command_reply: tid=0 2024-09-16T00:44:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 client.15783 shutdown 2024-09-16T00:44:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 -- 172.21.15.31:0/3209132842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 msgr2=0x5562f65dafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65dafe0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00fb50 tx=0x7f05bc004730 comp rx=0 tx=0).stop 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 -- 172.21.15.31:0/3209132842 shutdown_connections 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5562f65eb140 0x5562f65ed5c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 0x5562f65d87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 0x5562f65d8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209132842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65dafe0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 -- 172.21.15.31:0/3209132842 >> 172.21.15.31:0/3209132842 conn(0x5562f65cca30 msgr2=0x5562f65d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05aeff5640 0 client.15783 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 -- 172.21.15.31:0/3209132842 shutdown_connections 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 1 -- 172.21.15.31:0/3209132842 wait complete. 2024-09-16T00:44:08.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.323+0000 7f05d3981740 20 client.15783 trim_cache size 0 max 16384 2024-09-16T00:44:08.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.341+0000 7f05d3981740 1 Processor -- start 2024-09-16T00:44:08.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.341+0000 7f05d3981740 1 -- start start 2024-09-16T00:44:08.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05d3981740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05d3981740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 0x5562f65f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05d3981740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 0x5562f65f3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05d3981740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1b32c0 con 0x5562f65d04a0 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05d3981740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1b5b40 con 0x5562f65d0dc0 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05d3981740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1b3930 con 0x5562f65db680 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05caffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05caffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54508/0 (socket says 172.21.15.31:54508) 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05caffd640 1 -- 172.21.15.31:0/1577142308 learned_addr learned my addr 172.21.15.31:0/1577142308 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/1577142308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 0x5562f65f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05cb7fe640 1 --2- 172.21.15.31:0/1577142308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 0x5562f65f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05caffd640 1 -- 172.21.15.31:0/1577142308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 msgr2=0x5562f65f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05caffd640 1 --2- 172.21.15.31:0/1577142308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 0x5562f65f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05caffd640 1 -- 172.21.15.31:0/1577142308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 msgr2=0x5562f65f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05caffd640 1 --2- 172.21.15.31:0/1577142308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 0x5562f65f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05caffd640 1 -- 172.21.15.31:0/1577142308 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f65d7240 con 0x5562f65d04a0 2024-09-16T00:44:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.342+0000 7f05caffd640 1 --2- 172.21.15.31:0/1577142308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65cba60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f05bc01ea40 tx=0x7f05bc009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21d7e40d42529ce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05b0ff9640 1 -- 172.21.15.31:0/1577142308 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc01e5a0 con 0x5562f65d04a0 2024-09-16T00:44:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05b0ff9640 1 -- 172.21.15.31:0/1577142308 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05bc01e740 con 0x5562f65d04a0 2024-09-16T00:44:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05b0ff9640 1 -- 172.21.15.31:0/1577142308 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc003920 con 0x5562f65d04a0 2024-09-16T00:44:08.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05d3981740 1 -- 172.21.15.31:0/1577142308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 msgr2=0x5562f65cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05d3981740 1 --2- 172.21.15.31:0/1577142308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65cba60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f05bc01ea40 tx=0x7f05bc009fd0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05d3981740 1 -- 172.21.15.31:0/1577142308 shutdown_connections 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05d3981740 1 --2- 172.21.15.31:0/1577142308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65db680 0x5562f65f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05d3981740 1 --2- 172.21.15.31:0/1577142308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d0dc0 0x5562f65f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05d3981740 1 --2- 172.21.15.31:0/1577142308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65d04a0 0x5562f65cba60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.343+0000 7f05d3981740 1 -- 172.21.15.31:0/1577142308 >> 172.21.15.31:0/1577142308 conn(0x5562f65cca30 msgr2=0x5562f65cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 -- 172.21.15.31:0/1577142308 shutdown_connections 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 -- 172.21.15.31:0/1577142308 wait complete. 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 -- start start 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 Processor -- start 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65e3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 0x5562f65e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 0x5562f65e0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1b4000 con 0x5562f65db680 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1b5470 con 0x5562f65d04a0 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1b46d0 con 0x5562f65d0dc0 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.344+0000 7f05d3981740 10 client.0 _ll_register_callbacks cb 0x7f05bc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05caffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05caffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65e3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:51872/0 (socket says 172.21.15.31:51872) 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05caffd640 1 -- 172.21.15.31:0/3209459575 learned_addr learned my addr 172.21.15.31:0/3209459575 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 0x5562f65e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05cb7fe640 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 0x5562f65e0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05caffd640 1 -- 172.21.15.31:0/3209459575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 msgr2=0x5562f65e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05caffd640 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 0x5562f65e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05caffd640 1 -- 172.21.15.31:0/3209459575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 msgr2=0x5562f65e0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05caffd640 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 0x5562f65e0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05caffd640 1 -- 172.21.15.31:0/3209459575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f65d8b60 con 0x5562f65d04a0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05caffd640 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65e3640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f05bc01eae0 tx=0x7f05bc01ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f589768651ab9f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209459575 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc01e5a0 con 0x5562f65d04a0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05d3981740 1 -- 172.21.15.31:0/3209459575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05bc01e740 con 0x5562f65d04a0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209459575 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f05bc024040 con 0x5562f65d04a0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209459575 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f05bc04c050 con 0x5562f65d04a0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05d3981740 10 client.25472 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.345+0000 7f05d3981740 1 -- 172.21.15.31:0/3209459575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5562f66557f0 con 0x5562f65d04a0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.346+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209459575 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05bc01e740 con 0x5562f65d04a0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.346+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209459575 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f05bc005310 con 0x5562f65d04a0 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.346+0000 7f05d3981740 10 client.25472 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:08.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.346+0000 7f05d3981740 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:08.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.346+0000 7f05d3981740 20 client.25472 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:08.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.346+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5562f6655df0 0x5562f6658210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:08.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.346+0000 7f05d3981740 4 client.25472 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:44:08.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.346+0000 7f05d3981740 1 -- 172.21.15.31:0/3209459575 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562f64ba210 con 0x5562f6655df0 2024-09-16T00:44:08.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.346+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5562f6655df0 0x5562f6658210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:08.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.347+0000 7f05ca7fc640 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5562f6655df0 0x5562f6658210 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92916676c5a2b20 server_cookie=8209f73bb0673aa1 in_seq=0 out_seq=0 2024-09-16T00:44:08.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.347+0000 7f05aeff5640 10 client.25472 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:08.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05aeff5640 1 -- 172.21.15.31:0/3209459575 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5562f64ba210 con 0x5562f6655df0 2024-09-16T00:44:08.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05aeff5640 10 client.25472 handle_command_reply: tid=0 2024-09-16T00:44:08.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 client.25472 shutdown 2024-09-16T00:44:08.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 -- 172.21.15.31:0/3209459575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 msgr2=0x5562f65e3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65e3640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f05bc01eae0 tx=0x7f05bc01ede0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 -- 172.21.15.31:0/3209459575 shutdown_connections 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f65db680 0x5562f65e0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5562f6655df0 0x5562f6658210 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562f65d0dc0 0x5562f65e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05aeff5640 0 client.25472 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 --2- 172.21.15.31:0/3209459575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562f65d04a0 0x5562f65e3640 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 -- 172.21.15.31:0/3209459575 >> 172.21.15.31:0/3209459575 conn(0x5562f65cca30 msgr2=0x5562f65cbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 -- 172.21.15.31:0/3209459575 shutdown_connections 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.350+0000 7f05d3981740 1 -- 172.21.15.31:0/3209459575 wait complete. 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d3981740 20 client.25472 trim_cache size 0 max 16384 2024-09-16T00:44:08.353 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:44:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (5855 inodes in the stack)", 2024-09-16T00:44:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:44:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "e4ab83eb-50aa-440e-9051-fc9a63771276": { 2024-09-16T00:44:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:44:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "e4ab83eb-50aa-440e-9051-fc9a63771276", 2024-09-16T00:44:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:44:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:44:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:44:08.354 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 -- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05b40a3240 msgr2=0x7f05b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05b40a3240 0x7f05b40a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5562f65cd890 tx=0x7f05bc005f90 comp rx=0 tx=0).stop 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 -- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 msgr2=0x7f05cc1a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 0x7f05cc1a4c90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f05c0006010 tx=0x7f05c0031b20 comp rx=0 tx=0).stop 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 -- 172.21.15.31:0/1099022072 shutdown_connections 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05cc105f40 0x7f05cc19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc101450 0x7f05cc1a4c90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f05b40a3240 0x7f05b40a5700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 --2- 172.21.15.31:0/1099022072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05cc100b30 0x7f05cc1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 -- 172.21.15.31:0/1099022072 >> 172.21.15.31:0/1099022072 conn(0x7f05cc0fc7a0 msgr2=0x7f05cc10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 -- 172.21.15.31:0/1099022072 shutdown_connections 2024-09-16T00:44:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:08.351+0000 7f05d15a8640 1 -- 172.21.15.31:0/1099022072 wait complete. 2024-09-16T00:44:09.512 INFO:tasks.cephfs.filesystem:scrub status for tag:e4ab83eb-50aa-440e-9051-fc9a63771276 - {'path': '/', 'tag': 'e4ab83eb-50aa-440e-9051-fc9a63771276', 'options': 'recursive,force'} 2024-09-16T00:44:09.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:09 smithi081 ceph-mon[35539]: pgmap v446: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 169 KiB/s wr, 64 op/s 2024-09-16T00:44:09.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:09 smithi102 ceph-mon[35364]: pgmap v446: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 169 KiB/s wr, 64 op/s 2024-09-16T00:44:09.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:09 smithi031 ceph-mon[30956]: pgmap v446: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 169 KiB/s wr, 64 op/s 2024-09-16T00:44:11.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:11 smithi081 ceph-mon[35539]: pgmap v447: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 285 KiB/s wr, 52 op/s 2024-09-16T00:44:11.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:11 smithi102 ceph-mon[35364]: pgmap v447: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 285 KiB/s wr, 52 op/s 2024-09-16T00:44:11.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:11 smithi031 ceph-mon[30956]: pgmap v447: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 285 KiB/s wr, 52 op/s 2024-09-16T00:44:13.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:13 smithi081 ceph-mon[35539]: pgmap v448: 129 pgs: 129 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 997 KiB/s wr, 78 op/s 2024-09-16T00:44:13.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:13 smithi102 ceph-mon[35364]: pgmap v448: 129 pgs: 129 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 997 KiB/s wr, 78 op/s 2024-09-16T00:44:13.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:13 smithi031 ceph-mon[30956]: pgmap v448: 129 pgs: 129 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 997 KiB/s wr, 78 op/s 2024-09-16T00:44:15.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:15 smithi081 ceph-mon[35539]: pgmap v449: 129 pgs: 129 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 122 op/s 2024-09-16T00:44:15.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:15 smithi102 ceph-mon[35364]: pgmap v449: 129 pgs: 129 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 122 op/s 2024-09-16T00:44:15.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:15 smithi031 ceph-mon[30956]: pgmap v449: 129 pgs: 129 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 122 op/s 2024-09-16T00:44:16.187 INFO:tasks.workunit.client.0.smithi031.stdout: 59 1231334 507 774685 419 648815 1248 2024-09-16T00:44:17.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:17 smithi081 ceph-mon[35539]: pgmap v450: 129 pgs: 129 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.1 MiB/s wr, 125 op/s 2024-09-16T00:44:17.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:17 smithi102 ceph-mon[35364]: pgmap v450: 129 pgs: 129 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.1 MiB/s wr, 125 op/s 2024-09-16T00:44:17.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:17 smithi031 ceph-mon[30956]: pgmap v450: 129 pgs: 129 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.1 MiB/s wr, 125 op/s 2024-09-16T00:44:19.678 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:19 smithi031 ceph-mon[30956]: pgmap v451: 129 pgs: 129 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-09-16T00:44:19.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:19 smithi081 ceph-mon[35539]: pgmap v451: 129 pgs: 129 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-09-16T00:44:19.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:19 smithi102 ceph-mon[35364]: pgmap v451: 129 pgs: 129 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-09-16T00:44:21.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:21 smithi081 ceph-mon[35539]: pgmap v452: 129 pgs: 129 active+clean; 290 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-16T00:44:21.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:21 smithi102 ceph-mon[35364]: pgmap v452: 129 pgs: 129 active+clean; 290 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-16T00:44:21.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:21 smithi031 ceph-mon[30956]: pgmap v452: 129 pgs: 129 active+clean; 290 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-16T00:44:23.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:23 smithi081 ceph-mon[35539]: pgmap v453: 129 pgs: 129 active+clean; 295 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 137 op/s 2024-09-16T00:44:23.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:23 smithi102 ceph-mon[35364]: pgmap v453: 129 pgs: 129 active+clean; 295 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 137 op/s 2024-09-16T00:44:23.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:23 smithi031 ceph-mon[30956]: pgmap v453: 129 pgs: 129 active+clean; 295 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 137 op/s 2024-09-16T00:44:25.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:25 smithi081 ceph-mon[35539]: pgmap v454: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 122 op/s 2024-09-16T00:44:25.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:25 smithi102 ceph-mon[35364]: pgmap v454: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 122 op/s 2024-09-16T00:44:25.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:25 smithi031 ceph-mon[30956]: pgmap v454: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 122 op/s 2024-09-16T00:44:26.187 INFO:tasks.workunit.client.0.smithi031.stdout: 64 1624460 490 1038418 350 863635 1299 2024-09-16T00:44:27.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:27 smithi081 ceph-mon[35539]: pgmap v455: 129 pgs: 129 active+clean; 308 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 139 op/s 2024-09-16T00:44:27.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:27 smithi102 ceph-mon[35364]: pgmap v455: 129 pgs: 129 active+clean; 308 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 139 op/s 2024-09-16T00:44:27.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:27 smithi031 ceph-mon[30956]: pgmap v455: 129 pgs: 129 active+clean; 308 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 139 op/s 2024-09-16T00:44:29.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:29 smithi081 ceph-mon[35539]: pgmap v456: 129 pgs: 129 active+clean; 332 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-09-16T00:44:29.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:29 smithi102 ceph-mon[35364]: pgmap v456: 129 pgs: 129 active+clean; 332 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-09-16T00:44:29.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:29 smithi031 ceph-mon[30956]: pgmap v456: 129 pgs: 129 active+clean; 332 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-09-16T00:44:31.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:31 smithi081 ceph-mon[35539]: pgmap v457: 129 pgs: 129 active+clean; 339 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 288 op/s 2024-09-16T00:44:31.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:31 smithi102 ceph-mon[35364]: pgmap v457: 129 pgs: 129 active+clean; 339 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 288 op/s 2024-09-16T00:44:31.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:31 smithi031 ceph-mon[30956]: pgmap v457: 129 pgs: 129 active+clean; 339 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 288 op/s 2024-09-16T00:44:33.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:33 smithi081 ceph-mon[35539]: pgmap v458: 129 pgs: 129 active+clean; 354 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 376 op/s 2024-09-16T00:44:33.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:33 smithi102 ceph-mon[35364]: pgmap v458: 129 pgs: 129 active+clean; 354 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 376 op/s 2024-09-16T00:44:33.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:33 smithi031 ceph-mon[30956]: pgmap v458: 129 pgs: 129 active+clean; 354 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 376 op/s 2024-09-16T00:44:35.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:35 smithi081 ceph-mon[35539]: pgmap v459: 129 pgs: 129 active+clean; 367 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 467 op/s 2024-09-16T00:44:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:35 smithi102 ceph-mon[35364]: pgmap v459: 129 pgs: 129 active+clean; 367 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 467 op/s 2024-09-16T00:44:35.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:35 smithi031 ceph-mon[30956]: pgmap v459: 129 pgs: 129 active+clean; 367 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 467 op/s 2024-09-16T00:44:36.187 INFO:tasks.workunit.client.0.smithi031.stdout: 69 1662860 504 1086135 273 936745 1463 2024-09-16T00:44:37.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:37 smithi081 ceph-mon[35539]: pgmap v460: 129 pgs: 129 active+clean; 366 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 466 op/s 2024-09-16T00:44:37.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:37 smithi102 ceph-mon[35364]: pgmap v460: 129 pgs: 129 active+clean; 366 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 466 op/s 2024-09-16T00:44:37.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:37 smithi031 ceph-mon[30956]: pgmap v460: 129 pgs: 129 active+clean; 366 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 466 op/s 2024-09-16T00:44:39.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:44:39.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:39 smithi081 ceph-mon[35539]: pgmap v461: 129 pgs: 129 active+clean; 368 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 435 op/s 2024-09-16T00:44:39.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:39 smithi102 ceph-mon[35364]: pgmap v461: 129 pgs: 129 active+clean; 368 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 435 op/s 2024-09-16T00:44:39.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:39 smithi031 ceph-mon[30956]: pgmap v461: 129 pgs: 129 active+clean; 368 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 435 op/s 2024-09-16T00:44:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:44:41.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.287+0000 7f90669fa640 1 -- 172.21.15.31:0/1376841247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f906011b140 msgr2=0x7f906011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:41.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.287+0000 7f90669fa640 1 --2- 172.21.15.31:0/1376841247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f906011b140 0x7f906011d530 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9050009900 tx=0x7f905002a420 comp rx=0 tx=0).stop 2024-09-16T00:44:41.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.287+0000 7f90669fa640 1 -- 172.21.15.31:0/1376841247 shutdown_connections 2024-09-16T00:44:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.287+0000 7f90669fa640 1 --2- 172.21.15.31:0/1376841247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f906011b140 0x7f906011d530 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.287+0000 7f90669fa640 1 --2- 172.21.15.31:0/1376841247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9060109830 0x7f9060109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.287+0000 7f90669fa640 1 --2- 172.21.15.31:0/1376841247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9060109450 0x7f9060110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.287+0000 7f90669fa640 1 -- 172.21.15.31:0/1376841247 >> 172.21.15.31:0/1376841247 conn(0x7f906006e140 msgr2=0x7f906006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.291+0000 7f90669fa640 1 -- 172.21.15.31:0/1376841247 shutdown_connections 2024-09-16T00:44:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.292+0000 7f90669fa640 1 -- 172.21.15.31:0/1376841247 wait complete. 2024-09-16T00:44:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.293+0000 7f90669fa640 1 Processor -- start 2024-09-16T00:44:41.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.294+0000 7f90669fa640 1 -- start start 2024-09-16T00:44:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.295+0000 7f90669fa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9060109450 0x7f90601a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.295+0000 7f90669fa640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9060109830 0x7f90601a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.295+0000 7f90669fa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f906011b140 0x7f90601abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.295+0000 7f90669fa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9060120dc0 con 0x7f9060109450 2024-09-16T00:44:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.295+0000 7f90669fa640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9060120c40 con 0x7f906011b140 2024-09-16T00:44:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.295+0000 7f90669fa640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9060120f40 con 0x7f9060109830 2024-09-16T00:44:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.296+0000 7f905ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9060109450 0x7f90601a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.296+0000 7f905ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9060109450 0x7f90601a72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50454/0 (socket says 172.21.15.31:50454) 2024-09-16T00:44:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.296+0000 7f905ffff640 1 -- 172.21.15.31:0/634025233 learned_addr learned my addr 172.21.15.31:0/634025233 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.297+0000 7f905ffff640 1 -- 172.21.15.31:0/634025233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9060109830 msgr2=0x7f90601a7800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.297+0000 7f905ffff640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9060109830 0x7f90601a7800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.297+0000 7f905ffff640 1 -- 172.21.15.31:0/634025233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f906011b140 msgr2=0x7f90601abb90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.297+0000 7f905ffff640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f906011b140 0x7f90601abb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.297+0000 7f905ffff640 1 -- 172.21.15.31:0/634025233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90601ac270 con 0x7f9060109450 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.298+0000 7f905ffff640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9060109450 0x7f90601a72c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f905400b4a0 tx=0x7f905400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.298+0000 7f905d7fa640 1 -- 172.21.15.31:0/634025233 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054014070 con 0x7f9060109450 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.298+0000 7f905d7fa640 1 -- 172.21.15.31:0/634025233 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9054010af0 con 0x7f9060109450 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.298+0000 7f90669fa640 1 -- 172.21.15.31:0/634025233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90601ac560 con 0x7f9060109450 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.299+0000 7f90669fa640 1 -- 172.21.15.31:0/634025233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90601b3e40 con 0x7f9060109450 2024-09-16T00:44:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.301+0000 7f905d7fa640 1 -- 172.21.15.31:0/634025233 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054017c50 con 0x7f9060109450 2024-09-16T00:44:41.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.305+0000 7f9068dd3740 1 Processor -- start 2024-09-16T00:44:41.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.305+0000 7f9068dd3740 1 -- start start 2024-09-16T00:44:41.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.306+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.306+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 0x563b672fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.306+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b672fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.306+0000 7f9068dd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563b672572e0 con 0x563b672f14a0 2024-09-16T00:44:41.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.306+0000 7f9068dd3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563b671fece0 con 0x563b672f1dc0 2024-09-16T00:44:41.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.306+0000 7f9068dd3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563b672532a0 con 0x563b672fc680 2024-09-16T00:44:41.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.306+0000 7f905f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b672fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.306+0000 7f905f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b672fea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:54614/0 (socket says 172.21.15.31:54614) 2024-09-16T00:44:41.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.306+0000 7f905f7fe640 1 -- 172.21.15.31:0/30550867 learned_addr learned my addr 172.21.15.31:0/30550867 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:41.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.307+0000 7f9064f97640 1 --2- 172.21.15.31:0/30550867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.307+0000 7f9064f97640 1 -- 172.21.15.31:0/30550867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 msgr2=0x563b672fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:41.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.310+0000 7f9064f97640 1 --2- 172.21.15.31:0/30550867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b672fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.310+0000 7f9064f97640 1 -- 172.21.15.31:0/30550867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 msgr2=0x563b672fc140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:44:41.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.310+0000 7f9064f97640 1 --2- 172.21.15.31:0/30550867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 0x563b672fc140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.310+0000 7f9064f97640 1 -- 172.21.15.31:0/30550867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b671e3440 con 0x563b672f14a0 2024-09-16T00:44:41.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.311+0000 7f9064f97640 1 --2- 172.21.15.31:0/30550867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672f1880 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9050009850 tx=0x7f9050031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9614d5990b556eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:41.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.314+0000 7f905f7fe640 1 --2- 172.21.15.31:0/30550867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b672fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:41.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.314+0000 7f90467fc640 1 -- 172.21.15.31:0/30550867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905003e030 con 0x563b672f14a0 2024-09-16T00:44:41.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.314+0000 7f90467fc640 1 -- 172.21.15.31:0/30550867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9050031540 con 0x563b672f14a0 2024-09-16T00:44:41.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.315+0000 7f9068dd3740 1 -- 172.21.15.31:0/30550867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 msgr2=0x563b672f1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:41.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.315+0000 7f9068dd3740 1 --2- 172.21.15.31:0/30550867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672f1880 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9050009850 tx=0x7f9050031460 comp rx=0 tx=0).stop 2024-09-16T00:44:41.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.318+0000 7f905d7fa640 1 -- 172.21.15.31:0/634025233 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9054017070 con 0x7f9060109450 2024-09-16T00:44:41.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.319+0000 7f905d7fa640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90480a3450 0x7f90480a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.319+0000 7f905d7fa640 1 -- 172.21.15.31:0/634025233 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f90540cf010 con 0x7f9060109450 2024-09-16T00:44:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.320+0000 7f9068dd3740 1 -- 172.21.15.31:0/30550867 shutdown_connections 2024-09-16T00:44:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.320+0000 7f9068dd3740 1 --2- 172.21.15.31:0/30550867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b672fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.320+0000 7f9068dd3740 1 --2- 172.21.15.31:0/30550867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 0x563b672fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.320+0000 7f9068dd3740 1 --2- 172.21.15.31:0/30550867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672f1880 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.320+0000 7f9068dd3740 1 -- 172.21.15.31:0/30550867 >> 172.21.15.31:0/30550867 conn(0x563b672eda30 msgr2=0x563b672ede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.320+0000 7f9064f97640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90480a3450 0x7f90480a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.321+0000 7f9064f97640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90480a3450 0x7f90480a5910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9050005c40 tx=0x7f905002a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:41.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.326+0000 7f9068dd3740 1 -- 172.21.15.31:0/30550867 shutdown_connections 2024-09-16T00:44:41.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.328+0000 7f9068dd3740 1 -- 172.21.15.31:0/30550867 wait complete. 2024-09-16T00:44:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.328+0000 7f9068dd3740 1 -- start start 2024-09-16T00:44:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.329+0000 7f9068dd3740 1 Processor -- start 2024-09-16T00:44:41.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.336+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.336+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 0x563b672f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.336+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b672f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.336+0000 7f9068dd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563b672ff2a0 con 0x563b672f14a0 2024-09-16T00:44:41.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.336+0000 7f9068dd3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563b67212050 con 0x563b672f1dc0 2024-09-16T00:44:41.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.336+0000 7f9068dd3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563b672ff420 con 0x563b672fc680 2024-09-16T00:44:41.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.336+0000 7f9068dd3740 10 client.0 _ll_register_callbacks cb 0x563b67252160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:41.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.337+0000 7f905ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.337+0000 7f905ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672fbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50472/0 (socket says 172.21.15.31:50472) 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.337+0000 7f905ffff640 1 -- 172.21.15.31:0/1937942306 learned_addr learned my addr 172.21.15.31:0/1937942306 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.338+0000 7f905ffff640 1 -- 172.21.15.31:0/1937942306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 msgr2=0x563b672f96e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.338+0000 7f905ffff640 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b672f96e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.338+0000 7f905ffff640 1 -- 172.21.15.31:0/1937942306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 msgr2=0x563b672f91a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.338+0000 7f905ffff640 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 0x563b672f91a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.338+0000 7f905ffff640 1 -- 172.21.15.31:0/1937942306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b66fc83c0 con 0x563b672f14a0 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.338+0000 7f905ffff640 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672fbef0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f905409af20 tx=0x7f905409ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c9a8055f472014 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.338+0000 7f903f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.338+0000 7f903f7fe640 1 -- 172.21.15.31:0/1937942306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905401e070 con 0x563b672f14a0 2024-09-16T00:44:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.338+0000 7f9068dd3740 1 -- 172.21.15.31:0/1937942306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9050031540 con 0x563b672f14a0 2024-09-16T00:44:41.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.339+0000 7f903f7fe640 1 -- 172.21.15.31:0/1937942306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f90540cc810 con 0x563b672f14a0 2024-09-16T00:44:41.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.339+0000 7f903f7fe640 1 -- 172.21.15.31:0/1937942306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f90540b9e60 con 0x563b672f14a0 2024-09-16T00:44:41.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.339+0000 7f9068dd3740 10 client.15801 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:41.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.339+0000 7f9068dd3740 1 -- 172.21.15.31:0/1937942306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563b6731bc60 con 0x563b672f14a0 2024-09-16T00:44:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.344+0000 7f903f7fe640 1 -- 172.21.15.31:0/1937942306 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9050031540 con 0x563b672f14a0 2024-09-16T00:44:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.344+0000 7f903f7fe640 1 -- 172.21.15.31:0/1937942306 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f90540bb0b0 con 0x563b672f14a0 2024-09-16T00:44:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.345+0000 7f9068dd3740 10 client.15801 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.345+0000 7f9068dd3740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.345+0000 7f9068dd3740 20 client.15801 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.345+0000 7f9068dd3740 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563b6731d120 0x563b6731f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.345+0000 7f9068dd3740 4 client.15801 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:44:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.345+0000 7f9068dd3740 1 -- 172.21.15.31:0/1937942306 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b67264850 con 0x563b6731d120 2024-09-16T00:44:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.347+0000 7f905f7fe640 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563b6731d120 0x563b6731f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.348+0000 7f905f7fe640 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563b6731d120 0x563b6731f5e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577f173fd3a3d8d9 server_cookie=acdbfbab0961feee in_seq=0 out_seq=0 2024-09-16T00:44:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.349+0000 7f903f7fe640 10 client.15801 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.352+0000 7f903f7fe640 1 -- 172.21.15.31:0/1937942306 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b67264850 con 0x563b6731d120 2024-09-16T00:44:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.352+0000 7f903f7fe640 10 client.15801 handle_command_reply: tid=0 2024-09-16T00:44:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.353+0000 7f9068dd3740 1 client.15801 shutdown 2024-09-16T00:44:41.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.359+0000 7f9068dd3740 1 -- 172.21.15.31:0/1937942306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 msgr2=0x563b672fbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:41.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.359+0000 7f9068dd3740 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672fbef0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f905409af20 tx=0x7f905409ba60 comp rx=0 tx=0).stop 2024-09-16T00:44:41.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.366+0000 7f9068dd3740 1 -- 172.21.15.31:0/1937942306 shutdown_connections 2024-09-16T00:44:41.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.366+0000 7f9068dd3740 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563b6731d120 0x563b6731f5e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.366+0000 7f9068dd3740 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b672f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.366+0000 7f9068dd3740 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 0x563b672f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.366+0000 7f9068dd3740 1 --2- 172.21.15.31:0/1937942306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b672fbef0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.366+0000 7f9068dd3740 1 -- 172.21.15.31:0/1937942306 >> 172.21.15.31:0/1937942306 conn(0x563b672eda30 msgr2=0x563b672f3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:41.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.367+0000 7f903f7fe640 0 client.15801 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:41.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.371+0000 7f9068dd3740 1 -- 172.21.15.31:0/1937942306 shutdown_connections 2024-09-16T00:44:41.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.374+0000 7f9068dd3740 1 -- 172.21.15.31:0/1937942306 wait complete. 2024-09-16T00:44:41.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.374+0000 7f9068dd3740 20 client.15801 trim_cache size 0 max 16384 2024-09-16T00:44:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.419+0000 7f9068dd3740 1 Processor -- start 2024-09-16T00:44:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.419+0000 7f9068dd3740 1 -- start start 2024-09-16T00:44:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.420+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67324f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.420+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672f1dc0 0x563b67325480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.420+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672fc680 0x563b672f6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.420+0000 7f9068dd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00dd70 con 0x563b672f14a0 2024-09-16T00:44:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.420+0000 7f9068dd3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0105f0 con 0x563b672fc680 2024-09-16T00:44:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.420+0000 7f9068dd3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00e3e0 con 0x563b672f1dc0 2024-09-16T00:44:41.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.421+0000 7f905f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672f1dc0 0x563b67325480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.421+0000 7f905f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672f1dc0 0x563b67325480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:54634/0 (socket says 172.21.15.31:54634) 2024-09-16T00:44:41.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.421+0000 7f905f7fe640 1 -- 172.21.15.31:0/2167466048 learned_addr learned my addr 172.21.15.31:0/2167466048 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:41.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.422+0000 7f9064f97640 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672fc680 0x563b672f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.421+0000 7f905ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67324f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.423+0000 7f905f7fe640 1 -- 172.21.15.31:0/2167466048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672fc680 msgr2=0x563b672f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.426+0000 7f905f7fe640 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672fc680 0x563b672f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.426+0000 7f905f7fe640 1 -- 172.21.15.31:0/2167466048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 msgr2=0x563b67324f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.427+0000 7f905f7fe640 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67324f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.427+0000 7f905f7fe640 1 -- 172.21.15.31:0/2167466048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6731fc60 con 0x563b672f1dc0 2024-09-16T00:44:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.427+0000 7f905ffff640 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67324f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.428+0000 7f9064f97640 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672fc680 0x563b672f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.430+0000 7f905f7fe640 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672f1dc0 0x563b67325480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x563b672fa7e0 tx=0x7f9058009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebccdc217de08a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.431+0000 7f9045ffb640 1 -- 172.21.15.31:0/2167466048 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058004670 con 0x563b672f1dc0 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.431+0000 7f9045ffb640 1 -- 172.21.15.31:0/2167466048 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9058004810 con 0x563b672f1dc0 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.432+0000 7f9068dd3740 1 -- 172.21.15.31:0/2167466048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672f1dc0 msgr2=0x563b67325480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.432+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672f1dc0 0x563b67325480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x563b672fa7e0 tx=0x7f9058009500 comp rx=0 tx=0).stop 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.433+0000 7f9068dd3740 1 -- 172.21.15.31:0/2167466048 shutdown_connections 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.433+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672fc680 0x563b672f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.433+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672f1dc0 0x563b67325480 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.433+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2167466048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67324f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.433+0000 7f9068dd3740 1 -- 172.21.15.31:0/2167466048 >> 172.21.15.31:0/2167466048 conn(0x563b672eda30 msgr2=0x563b672f4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:41.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.438+0000 7f9068dd3740 1 -- 172.21.15.31:0/2167466048 shutdown_connections 2024-09-16T00:44:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.438+0000 7f9068dd3740 1 -- 172.21.15.31:0/2167466048 wait complete. 2024-09-16T00:44:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.438+0000 7f9068dd3740 1 -- start start 2024-09-16T00:44:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.439+0000 7f9068dd3740 1 Processor -- start 2024-09-16T00:44:41.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.439+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67301050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.439+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 0x563b67301590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.439+0000 7f9068dd3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b6737fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.439+0000 7f9068dd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00eab0 con 0x563b672f14a0 2024-09-16T00:44:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.439+0000 7f9068dd3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00ff20 con 0x563b672f1dc0 2024-09-16T00:44:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.439+0000 7f9068dd3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00f180 con 0x563b672fc680 2024-09-16T00:44:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.439+0000 7f9068dd3740 10 client.0 _ll_register_callbacks cb 0x7f90540974c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:44:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.440+0000 7f905ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67301050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.440+0000 7f905ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67301050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50492/0 (socket says 172.21.15.31:50492) 2024-09-16T00:44:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.440+0000 7f905ffff640 1 -- 172.21.15.31:0/2221839652 learned_addr learned my addr 172.21.15.31:0/2221839652 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:44:41.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.440+0000 7f9064f97640 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b6737fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.440+0000 7f905ffff640 1 -- 172.21.15.31:0/2221839652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 msgr2=0x563b6737fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:44:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.440+0000 7f905ffff640 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b6737fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.440+0000 7f905ffff640 1 -- 172.21.15.31:0/2221839652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 msgr2=0x563b67301590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:44:41.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.441+0000 7f905ffff640 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 0x563b67301590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.441+0000 7f905ffff640 1 -- 172.21.15.31:0/2221839652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b672ed860 con 0x563b672f14a0 2024-09-16T00:44:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.441+0000 7f905ffff640 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67301050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f905409b1b0 tx=0x7f9054017360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c811e28bfc1c7e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.443+0000 7f903f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:44:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.443+0000 7f903f7fe640 1 -- 172.21.15.31:0/2221839652 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905401e050 con 0x563b672f14a0 2024-09-16T00:44:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.443+0000 7f903f7fe640 1 -- 172.21.15.31:0/2221839652 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f9054010ca0 con 0x563b672f14a0 2024-09-16T00:44:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.443+0000 7f9068dd3740 1 -- 172.21.15.31:0/2221839652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9058004810 con 0x563b672f14a0 2024-09-16T00:44:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.447+0000 7f903f7fe640 1 -- 172.21.15.31:0/2221839652 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f90540a6030 con 0x563b672f14a0 2024-09-16T00:44:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.451+0000 7f9068dd3740 10 client.15804 fetch_fsmap learned FSMap version 20 2024-09-16T00:44:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.451+0000 7f9068dd3740 1 -- 172.21.15.31:0/2221839652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563b67388850 con 0x563b672f14a0 2024-09-16T00:44:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.452+0000 7f903f7fe640 1 -- 172.21.15.31:0/2221839652 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9058004810 con 0x563b672f14a0 2024-09-16T00:44:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.452+0000 7f903f7fe640 1 -- 172.21.15.31:0/2221839652 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f905409f120 con 0x563b672f14a0 2024-09-16T00:44:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.455+0000 7f9068dd3740 10 client.15804 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:44:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.455+0000 7f9068dd3740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:44:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.455+0000 7f9068dd3740 20 client.15804 populate_metadata read hostname 'smithi031' 2024-09-16T00:44:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.455+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563b67388fa0 0x563b6738b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.455+0000 7f9068dd3740 4 client.15804 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:44:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.455+0000 7f9068dd3740 1 -- 172.21.15.31:0/2221839652 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b6725ed60 con 0x563b67388fa0 2024-09-16T00:44:41.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.456+0000 7f905f7fe640 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563b67388fa0 0x563b6738b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.465+0000 7f905f7fe640 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563b67388fa0 0x563b6738b3c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18d23a5ec0a3d793 server_cookie=4d36e9996b6a68ac in_seq=0 out_seq=0 2024-09-16T00:44:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.466+0000 7f903f7fe640 10 client.15804 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:41.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.469+0000 7f903f7fe640 1 -- 172.21.15.31:0/2221839652 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x563b6725ed60 con 0x563b67388fa0 2024-09-16T00:44:41.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.469+0000 7f903f7fe640 10 client.15804 handle_command_reply: tid=0 2024-09-16T00:44:41.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.470+0000 7f9068dd3740 1 client.15804 shutdown 2024-09-16T00:44:41.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.473+0000 7f9068dd3740 1 -- 172.21.15.31:0/2221839652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 msgr2=0x563b67301050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:41.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.473+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67301050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f905409b1b0 tx=0x7f9054017360 comp rx=0 tx=0).stop 2024-09-16T00:44:41.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.474+0000 7f9068dd3740 1 -- 172.21.15.31:0/2221839652 shutdown_connections 2024-09-16T00:44:41.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.474+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563b67388fa0 0x563b6738b3c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.474+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563b672fc680 0x563b6737fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.474+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b672f1dc0 0x563b67301590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.474+0000 7f9068dd3740 1 --2- 172.21.15.31:0/2221839652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b672f14a0 0x563b67301050 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.474+0000 7f9068dd3740 1 -- 172.21.15.31:0/2221839652 >> 172.21.15.31:0/2221839652 conn(0x563b672eda30 msgr2=0x563b673075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:41.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.478+0000 7f903f7fe640 0 client.15804 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:44:41.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.482+0000 7f9068dd3740 1 -- 172.21.15.31:0/2221839652 shutdown_connections 2024-09-16T00:44:41.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.482+0000 7f9068dd3740 1 -- 172.21.15.31:0/2221839652 wait complete. 2024-09-16T00:44:41.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.482+0000 7f9068dd3740 20 client.15804 trim_cache size 0 max 16384 2024-09-16T00:44:41.486 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:44:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (1750 inodes in the stack)", 2024-09-16T00:44:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:44:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "e4ab83eb-50aa-440e-9051-fc9a63771276": { 2024-09-16T00:44:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:44:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "e4ab83eb-50aa-440e-9051-fc9a63771276", 2024-09-16T00:44:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:44:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:44:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:44:41.487 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:44:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.486+0000 7f90669fa640 1 -- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90480a3450 msgr2=0x7f90480a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.487+0000 7f90669fa640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90480a3450 0x7f90480a5910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9050005c40 tx=0x7f905002a420 comp rx=0 tx=0).stop 2024-09-16T00:44:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.487+0000 7f90669fa640 1 -- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9060109450 msgr2=0x7f90601a72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.487+0000 7f90669fa640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9060109450 0x7f90601a72c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f905400b4a0 tx=0x7f905400b960 comp rx=0 tx=0).stop 2024-09-16T00:44:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.487+0000 7f90669fa640 1 -- 172.21.15.31:0/634025233 shutdown_connections 2024-09-16T00:44:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.487+0000 7f90669fa640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f906011b140 0x7f90601abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.487+0000 7f90669fa640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90480a3450 0x7f90480a5910 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.487+0000 7f90669fa640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9060109830 0x7f90601a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.487+0000 7f90669fa640 1 --2- 172.21.15.31:0/634025233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9060109450 0x7f90601a72c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:41.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.487+0000 7f90669fa640 1 -- 172.21.15.31:0/634025233 >> 172.21.15.31:0/634025233 conn(0x7f906006e140 msgr2=0x7f906011bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.490+0000 7f90669fa640 1 -- 172.21.15.31:0/634025233 shutdown_connections 2024-09-16T00:44:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:44:41.490+0000 7f90669fa640 1 -- 172.21.15.31:0/634025233 wait complete. 2024-09-16T00:44:41.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:41 smithi081 ceph-mon[35539]: pgmap v462: 129 pgs: 129 active+clean; 371 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 321 op/s 2024-09-16T00:44:41.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:41 smithi102 ceph-mon[35364]: pgmap v462: 129 pgs: 129 active+clean; 371 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 321 op/s 2024-09-16T00:44:41.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:41 smithi031 ceph-mon[30956]: pgmap v462: 129 pgs: 129 active+clean; 371 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 321 op/s 2024-09-16T00:44:42.683 INFO:tasks.cephfs.filesystem:scrub status for tag:e4ab83eb-50aa-440e-9051-fc9a63771276 - {'path': '/', 'tag': 'e4ab83eb-50aa-440e-9051-fc9a63771276', 'options': 'recursive,force'} 2024-09-16T00:44:43.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:43 smithi081 ceph-mon[35539]: pgmap v463: 129 pgs: 129 active+clean; 376 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 293 op/s 2024-09-16T00:44:43.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:43 smithi102 ceph-mon[35364]: pgmap v463: 129 pgs: 129 active+clean; 376 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 293 op/s 2024-09-16T00:44:43.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:43 smithi031 ceph-mon[30956]: pgmap v463: 129 pgs: 129 active+clean; 376 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 293 op/s 2024-09-16T00:44:45.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:45 smithi081 ceph-mon[35539]: pgmap v464: 129 pgs: 129 active+clean; 375 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-16T00:44:45.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:45 smithi102 ceph-mon[35364]: pgmap v464: 129 pgs: 129 active+clean; 375 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-16T00:44:45.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:45 smithi031 ceph-mon[30956]: pgmap v464: 129 pgs: 129 active+clean; 375 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-16T00:44:46.187 INFO:tasks.workunit.client.0.smithi031.stdout: 75 1568141 472 1000208 299 918757 1860 2024-09-16T00:44:47.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:47 smithi081 ceph-mon[35539]: pgmap v465: 129 pgs: 129 active+clean; 379 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-09-16T00:44:47.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:47 smithi102 ceph-mon[35364]: pgmap v465: 129 pgs: 129 active+clean; 379 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-09-16T00:44:47.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:47 smithi031 ceph-mon[30956]: pgmap v465: 129 pgs: 129 active+clean; 379 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-09-16T00:44:49.747 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:49 smithi081 ceph-mon[35539]: pgmap v466: 129 pgs: 129 active+clean; 378 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-09-16T00:44:49.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:49 smithi102 ceph-mon[35364]: pgmap v466: 129 pgs: 129 active+clean; 378 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-09-16T00:44:49.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:49 smithi031 ceph-mon[30956]: pgmap v466: 129 pgs: 129 active+clean; 378 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-09-16T00:44:51.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:51 smithi102 ceph-mon[35364]: pgmap v467: 129 pgs: 129 active+clean; 385 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-09-16T00:44:51.877 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:51 smithi031 ceph-mon[30956]: pgmap v467: 129 pgs: 129 active+clean; 385 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-09-16T00:44:51.996 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:51 smithi081 ceph-mon[35539]: pgmap v467: 129 pgs: 129 active+clean; 385 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-09-16T00:44:52.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:52 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:52.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:52 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:52.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:52 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:44:53.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:53 smithi102 ceph-mon[35364]: pgmap v468: 129 pgs: 129 active+clean; 384 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-09-16T00:44:53.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:53 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:44:53.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:53 smithi031 ceph-mon[30956]: pgmap v468: 129 pgs: 129 active+clean; 384 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-09-16T00:44:53.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:53 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:44:53.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:53 smithi081 ceph-mon[35539]: pgmap v468: 129 pgs: 129 active+clean; 384 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-09-16T00:44:53.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:53 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:44:55.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:55 smithi102 ceph-mon[35364]: pgmap v469: 129 pgs: 129 active+clean; 390 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-16T00:44:55.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:55 smithi031 ceph-mon[30956]: pgmap v469: 129 pgs: 129 active+clean; 390 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-16T00:44:55.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:55 smithi081 ceph-mon[35539]: pgmap v469: 129 pgs: 129 active+clean; 390 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-16T00:44:56.187 INFO:tasks.workunit.client.0.smithi031.stdout: 80 1012428 377 631541 321 590226 1269 2024-09-16T00:44:57.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:57 smithi102 ceph-mon[35364]: pgmap v470: 129 pgs: 129 active+clean; 401 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 172 op/s 2024-09-16T00:44:57.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:57 smithi031 ceph-mon[30956]: pgmap v470: 129 pgs: 129 active+clean; 401 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 172 op/s 2024-09-16T00:44:57.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:57 smithi081 ceph-mon[35539]: pgmap v470: 129 pgs: 129 active+clean; 401 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 172 op/s 2024-09-16T00:44:58.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:58 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:44:58.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:58 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:44:58.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:58 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:44:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:59 smithi102 ceph-mon[35364]: pgmap v471: 129 pgs: 129 active+clean; 444 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 495 op/s 2024-09-16T00:44:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:44:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:44:59.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:44:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:44:59.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:59 smithi031 ceph-mon[30956]: pgmap v471: 129 pgs: 129 active+clean; 444 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 495 op/s 2024-09-16T00:44:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:44:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:44:59.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:44:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:44:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:59 smithi081 ceph-mon[35539]: pgmap v471: 129 pgs: 129 active+clean; 444 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 495 op/s 2024-09-16T00:44:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:44:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:44:59.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:44:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:45:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:01 smithi102 ceph-mon[35364]: pgmap v472: 129 pgs: 129 active+clean; 463 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.0 MiB/s wr, 591 op/s 2024-09-16T00:45:01.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:01 smithi031 ceph-mon[30956]: pgmap v472: 129 pgs: 129 active+clean; 463 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.0 MiB/s wr, 591 op/s 2024-09-16T00:45:01.996 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:01 smithi081 ceph-mon[35539]: pgmap v472: 129 pgs: 129 active+clean; 463 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.0 MiB/s wr, 591 op/s 2024-09-16T00:45:03.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:03 smithi102 ceph-mon[35364]: pgmap v473: 129 pgs: 129 active+clean; 477 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.9 MiB/s wr, 704 op/s 2024-09-16T00:45:03.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:03 smithi031 ceph-mon[30956]: pgmap v473: 129 pgs: 129 active+clean; 477 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.9 MiB/s wr, 704 op/s 2024-09-16T00:45:03.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:03 smithi081 ceph-mon[35539]: pgmap v473: 129 pgs: 129 active+clean; 477 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.9 MiB/s wr, 704 op/s 2024-09-16T00:45:05.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:05 smithi102 ceph-mon[35364]: pgmap v474: 129 pgs: 129 active+clean; 481 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.6 MiB/s wr, 779 op/s 2024-09-16T00:45:05.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:05 smithi031 ceph-mon[30956]: pgmap v474: 129 pgs: 129 active+clean; 481 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.6 MiB/s wr, 779 op/s 2024-09-16T00:45:05.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:05 smithi081 ceph-mon[35539]: pgmap v474: 129 pgs: 129 active+clean; 481 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.6 MiB/s wr, 779 op/s 2024-09-16T00:45:06.187 INFO:tasks.workunit.client.0.smithi031.stdout: 84 1000519 272 626419 273 587348 1046 2024-09-16T00:45:07.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:07 smithi031 ceph-mon[30956]: pgmap v475: 129 pgs: 129 active+clean; 484 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 767 op/s 2024-09-16T00:45:07.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:07 smithi081 ceph-mon[35539]: pgmap v475: 129 pgs: 129 active+clean; 484 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 767 op/s 2024-09-16T00:45:08.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:07 smithi102 ceph-mon[35364]: pgmap v475: 129 pgs: 129 active+clean; 484 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 767 op/s 2024-09-16T00:45:09.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:09 smithi031 ceph-mon[30956]: pgmap v476: 129 pgs: 129 active+clean; 481 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-09-16T00:45:09.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:09 smithi081 ceph-mon[35539]: pgmap v476: 129 pgs: 129 active+clean; 481 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-09-16T00:45:10.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:09 smithi102 ceph-mon[35364]: pgmap v476: 129 pgs: 129 active+clean; 481 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-09-16T00:45:11.851 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:11 smithi102 ceph-mon[35364]: pgmap v477: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 403 op/s 2024-09-16T00:45:11.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:11 smithi031 ceph-mon[30956]: pgmap v477: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 403 op/s 2024-09-16T00:45:11.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:11 smithi081 ceph-mon[35539]: pgmap v477: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 403 op/s 2024-09-16T00:45:12.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:45:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:13 smithi031 ceph-mon[30956]: pgmap v478: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-09-16T00:45:13.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:13 smithi081 ceph-mon[35539]: pgmap v478: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-09-16T00:45:14.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:13 smithi102 ceph-mon[35364]: pgmap v478: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-09-16T00:45:14.862 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.8 2024-09-16T00:45:14.863 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:45:15.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:15 smithi031 ceph-mon[30956]: pgmap v479: 129 pgs: 129 active+clean; 491 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 322 op/s 2024-09-16T00:45:15.970 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:15.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:15 smithi081 ceph-mon[35539]: pgmap v479: 129 pgs: 129 active+clean; 491 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 322 op/s 2024-09-16T00:45:16.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:15 smithi102 ceph-mon[35364]: pgmap v479: 129 pgs: 129 active+clean; 491 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 322 op/s 2024-09-16T00:45:16.187 INFO:tasks.workunit.client.0.smithi031.stdout: 90 1842002 420 1167757 431 1083670 1742 2024-09-16T00:45:16.885 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:45:16.885 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:45:16.885 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:45:16.885 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:45:16.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.758+0000 7f64f7cc7640 1 -- 172.21.15.31:0/3314555831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f0101df0 msgr2=0x7f64f01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:16.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.758+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/3314555831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f0101df0 0x7f64f01021d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009900 tx=0x7f64ec02a3f0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.759+0000 7f64f7cc7640 1 -- 172.21.15.31:0/3314555831 shutdown_connections 2024-09-16T00:45:16.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.759+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/3314555831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64f0106f20 0x7f64f010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.759+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/3314555831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f01027a0 0x7f64f01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.759+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/3314555831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f0101df0 0x7f64f01021d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.759+0000 7f64f7cc7640 1 -- 172.21.15.31:0/3314555831 >> 172.21.15.31:0/3314555831 conn(0x7f64f00fd8f0 msgr2=0x7f64f00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:16.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.759+0000 7f64f7cc7640 1 -- 172.21.15.31:0/3314555831 shutdown_connections 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.759+0000 7f64f7cc7640 1 -- 172.21.15.31:0/3314555831 wait complete. 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.760+0000 7f64f7cc7640 1 Processor -- start 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.760+0000 7f64f7cc7640 1 -- start start 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.761+0000 7f64f7cc7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f0101df0 0x7f64f01a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.761+0000 7f64f7cc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64f01027a0 0x7f64f01a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.761+0000 7f64f5a63640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f0101df0 0x7f64f01a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.761+0000 7f64f5a63640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f0101df0 0x7f64f01a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:40042/0 (socket says 172.21.15.31:40042) 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.761+0000 7f64f5a63640 1 -- 172.21.15.31:0/750255922 learned_addr learned my addr 172.21.15.31:0/750255922 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.761+0000 7f64f5262640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64f01027a0 0x7f64f01a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.762+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f0106f20 0x7f64f010dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.762+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0111f80 con 0x7f64f01027a0 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.762+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0111e00 con 0x7f64f0106f20 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.762+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0112100 con 0x7f64f0101df0 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.762+0000 7f64f6264640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f0106f20 0x7f64f010dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.763+0000 7f64f5262640 1 -- 172.21.15.31:0/750255922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f0101df0 msgr2=0x7f64f01a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.763+0000 7f64f5262640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f0101df0 0x7f64f01a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.763+0000 7f64f5262640 1 -- 172.21.15.31:0/750255922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f0106f20 msgr2=0x7f64f010dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.763+0000 7f64f5262640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f0106f20 0x7f64f010dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.763+0000 7f64f5262640 1 -- 172.21.15.31:0/750255922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f010e480 con 0x7f64f01027a0 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.764+0000 7f64f5262640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64f01027a0 0x7f64f01a4ce0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f64e000d930 tx=0x7f64e000dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.764+0000 7f64e6ffd640 1 -- 172.21.15.31:0/750255922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0017070 con 0x7f64f01027a0 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.764+0000 7f64e6ffd640 1 -- 172.21.15.31:0/750255922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e0009820 con 0x7f64f01027a0 2024-09-16T00:45:16.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.764+0000 7f64e6ffd640 1 -- 172.21.15.31:0/750255922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0004ca0 con 0x7f64f01027a0 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.765+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f010e770 con 0x7f64f01027a0 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.766+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f01ab5e0 con 0x7f64f01027a0 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.769+0000 7f64e6ffd640 1 -- 172.21.15.31:0/750255922 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f64e00099d0 con 0x7f64f01027a0 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.770+0000 7f64e6ffd640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64dc0a32c0 0x7f64dc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.770+0000 7f64e6ffd640 1 -- 172.21.15.31:0/750255922 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f64e00cf730 con 0x7f64f01027a0 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.770+0000 7f64f5a63640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64dc0a32c0 0x7f64dc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.771+0000 7f64f5a63640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64dc0a32c0 0x7f64dc0a5780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004750 tx=0x7f64ec0046e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.771+0000 7f64fa0a0740 1 Processor -- start 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.772+0000 7f64fa0a0740 1 -- start start 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.772+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef60244a0 0x564ef6024880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.772+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef602f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5a63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef60244a0 0x564ef6024880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5a63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef60244a0 0x564ef6024880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50046/0 (socket says 172.21.15.31:50046) 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5a63640 1 -- 172.21.15.31:0/959196036 learned_addr learned my addr 172.21.15.31:0/959196036 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef602f680 0x564ef6031a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64fa0a0740 1 -- 172.21.15.31:0/959196036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564ef5f8a2e0 con 0x564ef60244a0 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64fa0a0740 1 -- 172.21.15.31:0/959196036 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564ef5f31ce0 con 0x564ef602f680 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64fa0a0740 1 -- 172.21.15.31:0/959196036 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ef5f862a0 con 0x564ef6024dc0 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5262640 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef602f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f6264640 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef602f680 0x564ef6031a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5a63640 1 -- 172.21.15.31:0/959196036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 msgr2=0x564ef602f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5a63640 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef602f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5a63640 1 -- 172.21.15.31:0/959196036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef602f680 msgr2=0x564ef6031a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:16.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5a63640 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef602f680 0x564ef6031a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5a63640 1 -- 172.21.15.31:0/959196036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef5f16440 con 0x564ef60244a0 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.773+0000 7f64f5262640 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef602f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.774+0000 7f64f5a63640 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef60244a0 0x564ef6024880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005880 tx=0x7f64ec03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84af8165a9f4ab7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.774+0000 7f64dbfff640 1 -- 172.21.15.31:0/959196036 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec04e070 con 0x564ef60244a0 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.774+0000 7f64dbfff640 1 -- 172.21.15.31:0/959196036 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64ec031c80 con 0x564ef60244a0 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.774+0000 7f64dbfff640 1 -- 172.21.15.31:0/959196036 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec035020 con 0x564ef60244a0 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 -- 172.21.15.31:0/959196036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef60244a0 msgr2=0x564ef6024880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef60244a0 0x564ef6024880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005880 tx=0x7f64ec03f000 comp rx=0 tx=0).stop 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 -- 172.21.15.31:0/959196036 shutdown_connections 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef602f680 0x564ef6031a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef602f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/959196036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef60244a0 0x564ef6024880 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 -- 172.21.15.31:0/959196036 >> 172.21.15.31:0/959196036 conn(0x564ef6020a30 msgr2=0x564ef6020e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:16.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 -- 172.21.15.31:0/959196036 shutdown_connections 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 -- 172.21.15.31:0/959196036 wait complete. 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 -- start start 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.775+0000 7f64fa0a0740 1 Processor -- start 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef60244a0 0x564ef602ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef602c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef602c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f6264640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef602c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f6264640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef602c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50050/0 (socket says 172.21.15.31:50050) 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f5262640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef602c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f6264640 1 -- 172.21.15.31:0/3260567693 learned_addr learned my addr 172.21.15.31:0/3260567693 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f5a63640 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef60244a0 0x564ef602ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564ef6032250 con 0x564ef602f680 2024-09-16T00:45:16.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564ef5f45050 con 0x564ef6024dc0 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ef60323d0 con 0x564ef60244a0 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64fa0a0740 10 client.0 _ll_register_callbacks cb 0x564ef5f85160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f5262640 1 -- 172.21.15.31:0/3260567693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef60244a0 msgr2=0x564ef602ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f5262640 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef60244a0 0x564ef602ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f5262640 1 -- 172.21.15.31:0/3260567693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 msgr2=0x564ef602c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f5262640 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef602c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.776+0000 7f64f5262640 1 -- 172.21.15.31:0/3260567693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef5cfb3c0 con 0x564ef6024dc0 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64f5262640 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef602c1d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64e00cced0 tx=0x7f64e009c1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1235e92759e4f040 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64d97fa640 1 -- 172.21.15.31:0/3260567693 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e00b3020 con 0x564ef6024dc0 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ec031c80 con 0x564ef6024dc0 2024-09-16T00:45:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64d97fa640 1 -- 172.21.15.31:0/3260567693 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f64e0009d40 con 0x564ef6024dc0 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64d97fa640 1 -- 172.21.15.31:0/3260567693 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f64e00a4610 con 0x564ef6024dc0 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64fa0a0740 10 client.25487 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564ef604ec10 con 0x564ef6024dc0 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64d97fa640 1 -- 172.21.15.31:0/3260567693 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64ec031c80 con 0x564ef6024dc0 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64d97fa640 1 -- 172.21.15.31:0/3260567693 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f64e00b4ba0 con 0x564ef6024dc0 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64fa0a0740 10 client.25487 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64fa0a0740 10 client.25487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.777+0000 7f64fa0a0740 20 client.25487 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.778+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564ef6050020 0x564ef6052460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.778+0000 7f64fa0a0740 4 client.25487 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.778+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef5f97850 con 0x564ef6050020 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.778+0000 7f64f5a63640 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564ef6050020 0x564ef6052460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.778+0000 7f64d97fa640 10 client.25487 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.778+0000 7f64f5a63640 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564ef6050020 0x564ef6052460 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74af6e77003e9355 server_cookie=e100fac40a9e8197 in_seq=0 out_seq=0 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.782+0000 7f64d97fa640 1 -- 172.21.15.31:0/3260567693 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ef5f97850 con 0x564ef6050020 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.782+0000 7f64d97fa640 10 client.25487 handle_command_reply: tid=0 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.782+0000 7f64fa0a0740 1 client.25487 shutdown 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.782+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 msgr2=0x564ef602c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.782+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef602c1d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64e00cced0 tx=0x7f64e009c1e0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 shutdown_connections 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef602c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef602c1d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564ef6050020 0x564ef6052460 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/3260567693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef60244a0 0x564ef602ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64d97fa640 0 client.25487 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:16.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 >> 172.21.15.31:0/3260567693 conn(0x564ef6020a30 msgr2=0x564ef601fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 shutdown_connections 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64fa0a0740 1 -- 172.21.15.31:0/3260567693 wait complete. 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.783+0000 7f64fa0a0740 20 client.25487 trim_cache size 0 max 16384 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.829+0000 7f64fa0a0740 1 Processor -- start 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.832+0000 7f64fa0a0740 1 -- start start 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.832+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef60244a0 0x564ef6057d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.832+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef6058270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.832+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef6029430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.832+0000 7f64fa0a0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f64f019ee60 con 0x564ef602f680 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.832+0000 7f64fa0a0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f64f006d750 con 0x564ef60244a0 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.832+0000 7f64fa0a0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64f019f4d0 con 0x564ef6024dc0 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f5262640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef6058270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f5262640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef6058270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:40056/0 (socket says 172.21.15.31:40056) 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f5262640 1 -- 172.21.15.31:0/350337498 learned_addr learned my addr 172.21.15.31:0/350337498 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f6264640 1 --2- 172.21.15.31:0/350337498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef6029430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f5262640 1 -- 172.21.15.31:0/350337498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef60244a0 msgr2=0x564ef6057d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f5262640 1 --2- 172.21.15.31:0/350337498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef60244a0 0x564ef6057d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f5262640 1 -- 172.21.15.31:0/350337498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 msgr2=0x564ef6029430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f5262640 1 --2- 172.21.15.31:0/350337498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef6029430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f5262640 1 -- 172.21.15.31:0/350337498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef6021ae0 con 0x564ef6024dc0 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.834+0000 7f64f5262640 1 --2- 172.21.15.31:0/350337498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef6058270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f64e009b6a0 tx=0x7f64e0099eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d96fbe4108c869f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.835+0000 7f64db7fe640 1 -- 172.21.15.31:0/350337498 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0015070 con 0x564ef6024dc0 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.835+0000 7f64db7fe640 1 -- 172.21.15.31:0/350337498 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e0098010 con 0x564ef6024dc0 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.835+0000 7f64db7fe640 1 -- 172.21.15.31:0/350337498 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e00b3050 con 0x564ef6024dc0 2024-09-16T00:45:16.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.836+0000 7f64fa0a0740 1 -- 172.21.15.31:0/350337498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 msgr2=0x564ef6058270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.836+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/350337498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef6058270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f64e009b6a0 tx=0x7f64e0099eb0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.842+0000 7f64fa0a0740 1 -- 172.21.15.31:0/350337498 shutdown_connections 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.842+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/350337498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef6029430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.842+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/350337498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef6024dc0 0x564ef6058270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.842+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/350337498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef60244a0 0x564ef6057d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.842+0000 7f64fa0a0740 1 -- 172.21.15.31:0/350337498 >> 172.21.15.31:0/350337498 conn(0x564ef6020a30 msgr2=0x564ef602e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.843+0000 7f64fa0a0740 1 -- 172.21.15.31:0/350337498 shutdown_connections 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.846+0000 7f64fa0a0740 1 -- 172.21.15.31:0/350337498 wait complete. 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.846+0000 7f64fa0a0740 1 -- start start 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.847+0000 7f64fa0a0740 1 Processor -- start 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.848+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef60244a0 0x564ef6044760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.848+0000 7f64fa0a0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef6044ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.848+0000 7f64f5262640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef6044ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.848+0000 7f64f5262640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef6044ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:47898/0 (socket says 172.21.15.31:47898) 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.848+0000 7f64f5262640 1 -- 172.21.15.31:0/591127517 learned_addr learned my addr 172.21.15.31:0/591127517 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.848+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef6045220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.848+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f64f006bc10 con 0x564ef602f680 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.849+0000 7f64f6264640 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef6045220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.849+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f64f006d080 con 0x564ef6024dc0 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.849+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64f006c2e0 con 0x564ef60244a0 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.849+0000 7f64fa0a0740 10 client.0 _ll_register_callbacks cb 0x7f64e009b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.849+0000 7f64f5262640 1 -- 172.21.15.31:0/591127517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef60244a0 msgr2=0x564ef6044760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.849+0000 7f64f5262640 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef60244a0 0x564ef6044760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.849+0000 7f64f5262640 1 -- 172.21.15.31:0/591127517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 msgr2=0x564ef6045220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.849+0000 7f64f5262640 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef6045220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.849+0000 7f64f5262640 1 -- 172.21.15.31:0/591127517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef60b9ff0 con 0x564ef6024dc0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.850+0000 7f64f5262640 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef6044ca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f64e000d900 tx=0x7f64e00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3761434e478f5c4c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.851+0000 7f64d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.851+0000 7f64d97fa640 1 -- 172.21.15.31:0/591127517 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e00a9030 con 0x564ef6024dc0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.851+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e0098010 con 0x564ef6024dc0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.851+0000 7f64d97fa640 1 -- 172.21.15.31:0/591127517 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f64e009b0c0 con 0x564ef6024dc0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.854+0000 7f64f6264640 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef6045220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.854+0000 7f64d97fa640 1 -- 172.21.15.31:0/591127517 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f64e00b3070 con 0x564ef6024dc0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.855+0000 7f64fa0a0740 10 client.25490 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.855+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564ef60b9ff0 con 0x564ef6024dc0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.857+0000 7f64d97fa640 1 -- 172.21.15.31:0/591127517 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e0098010 con 0x564ef6024dc0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.857+0000 7f64d97fa640 1 -- 172.21.15.31:0/591127517 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f64e0005220 con 0x564ef6024dc0 2024-09-16T00:45:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.857+0000 7f64fa0a0740 10 client.25490 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.857+0000 7f64fa0a0740 10 client.25490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.857+0000 7f64fa0a0740 20 client.25490 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.857+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564ef60bb570 0x564ef60bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.857+0000 7f64fa0a0740 4 client.25490 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.857+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ef5f91d60 con 0x564ef60bb570 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.858+0000 7f64f5a63640 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564ef60bb570 0x564ef60bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.868+0000 7f64f5a63640 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564ef60bb570 0x564ef60bd990 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f3ca37ccf48d6a server_cookie=a262121e1fa14a6d in_seq=0 out_seq=0 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.869+0000 7f64d97fa640 10 client.25490 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.874+0000 7f64d97fa640 1 -- 172.21.15.31:0/591127517 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564ef5f91d60 con 0x564ef60bb570 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.874+0000 7f64d97fa640 10 client.25490 handle_command_reply: tid=0 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.874+0000 7f64fa0a0740 1 client.25490 shutdown 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.875+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 msgr2=0x564ef6044ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.875+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef6044ca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f64e000d900 tx=0x7f64e00046c0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.875+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 shutdown_connections 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.875+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ef602f680 0x564ef6045220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.875+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564ef6024dc0 0x564ef6044ca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.875+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564ef60bb570 0x564ef60bd990 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.875+0000 7f64fa0a0740 1 --2- 172.21.15.31:0/591127517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef60244a0 0x564ef6044760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.875+0000 7f64d97fa640 0 client.25490 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.875+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 >> 172.21.15.31:0/591127517 conn(0x564ef6020a30 msgr2=0x564ef6025ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.876+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 shutdown_connections 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.878+0000 7f64fa0a0740 1 -- 172.21.15.31:0/591127517 wait complete. 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.882+0000 7f64fa0a0740 20 client.25490 trim_cache size 0 max 16384 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.883+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64dc0a32c0 msgr2=0x7f64dc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.883+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64dc0a32c0 0x7f64dc0a5780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004750 tx=0x7f64ec0046e0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.883+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64f01027a0 msgr2=0x7f64f01a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.883+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64f01027a0 0x7f64f01a4ce0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f64e000d930 tx=0x7f64e000dd30 comp rx=0 tx=0).stop 2024-09-16T00:45:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.887+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 shutdown_connections 2024-09-16T00:45:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.887+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f0106f20 0x7f64f010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.887+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64f01027a0 0x7f64f01a4ce0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.887+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64dc0a32c0 0x7f64dc0a5780 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.887+0000 7f64f7cc7640 1 --2- 172.21.15.31:0/750255922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f0101df0 0x7f64f01a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:16.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.887+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 >> 172.21.15.31:0/750255922 conn(0x7f64f00fd8f0 msgr2=0x7f64f00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:16.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.892+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 shutdown_connections 2024-09-16T00:45:16.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:16.892+0000 7f64f7cc7640 1 -- 172.21.15.31:0/750255922 wait complete. 2024-09-16T00:45:17.597 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:17 smithi031 ceph-mon[30956]: pgmap v480: 129 pgs: 129 active+clean; 490 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 297 op/s 2024-09-16T00:45:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:17 smithi081 ceph-mon[35539]: pgmap v480: 129 pgs: 129 active+clean; 490 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 297 op/s 2024-09-16T00:45:18.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:17 smithi102 ceph-mon[35364]: pgmap v480: 129 pgs: 129 active+clean; 490 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 297 op/s 2024-09-16T00:45:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:18.308 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:45:18.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:45:19.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.347+0000 7f615513b640 1 -- 172.21.15.31:0/3838262581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6150109830 msgr2=0x7f6150109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:19.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.347+0000 7f615513b640 1 --2- 172.21.15.31:0/3838262581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6150109830 0x7f6150109c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6148009ca0 tx=0x7f614802a8f0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.348+0000 7f615513b640 1 -- 172.21.15.31:0/3838262581 shutdown_connections 2024-09-16T00:45:19.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.348+0000 7f615513b640 1 --2- 172.21.15.31:0/3838262581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f615011b140 0x7f615011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.348+0000 7f615513b640 1 --2- 172.21.15.31:0/3838262581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6150109830 0x7f6150109c90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.348+0000 7f615513b640 1 --2- 172.21.15.31:0/3838262581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6150109450 0x7f6150110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.348+0000 7f615513b640 1 -- 172.21.15.31:0/3838262581 >> 172.21.15.31:0/3838262581 conn(0x7f615006e140 msgr2=0x7f615006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.349+0000 7f615513b640 1 -- 172.21.15.31:0/3838262581 shutdown_connections 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.349+0000 7f615513b640 1 -- 172.21.15.31:0/3838262581 wait complete. 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.350+0000 7f615513b640 1 Processor -- start 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.354+0000 7f615513b640 1 -- start start 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.355+0000 7f615513b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6150109450 0x7f61501a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.355+0000 7f615513b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615011b140 0x7f61501a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.355+0000 7f615513b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61501ab860 0x7f61501abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.355+0000 7f615513b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6150120df0 con 0x7f61501ab860 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.355+0000 7f615513b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6150120c70 con 0x7f615011b140 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.355+0000 7f615513b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6150120f70 con 0x7f6150109450 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.356+0000 7f614f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61501ab860 0x7f61501abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.356+0000 7f614f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61501ab860 0x7f61501abd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58516/0 (socket says 172.21.15.31:58516) 2024-09-16T00:45:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.356+0000 7f614f59e640 1 -- 172.21.15.31:0/3976205539 learned_addr learned my addr 172.21.15.31:0/3976205539 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.358+0000 7f614f59e640 1 -- 172.21.15.31:0/3976205539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6150109450 msgr2=0x7f61501a6f90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:45:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.358+0000 7f614f59e640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6150109450 0x7f61501a6f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.358+0000 7f614e59c640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615011b140 0x7f61501a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.358+0000 7f614f59e640 1 -- 172.21.15.31:0/3976205539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615011b140 msgr2=0x7f61501a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.358+0000 7f614f59e640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615011b140 0x7f61501a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.358+0000 7f614f59e640 1 -- 172.21.15.31:0/3976205539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61501ac3b0 con 0x7f61501ab860 2024-09-16T00:45:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.359+0000 7f614f59e640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61501ab860 0x7f61501abd10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f614000d720 tx=0x7f614000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:19.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.360+0000 7f614e59c640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615011b140 0x7f61501a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:19.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.366+0000 7f6137fff640 1 -- 172.21.15.31:0/3976205539 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140017030 con 0x7f61501ab860 2024-09-16T00:45:19.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.366+0000 7f6137fff640 1 -- 172.21.15.31:0/3976205539 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614000de30 con 0x7f61501ab860 2024-09-16T00:45:19.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.366+0000 7f6137fff640 1 -- 172.21.15.31:0/3976205539 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140004ba0 con 0x7f61501ab860 2024-09-16T00:45:19.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.368+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61501ac6a0 con 0x7f61501ab860 2024-09-16T00:45:19.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.369+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501b3e00 con 0x7f61501ab860 2024-09-16T00:45:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.374+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6150073b00 con 0x7f61501ab860 2024-09-16T00:45:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.381+0000 7f6137fff640 1 -- 172.21.15.31:0/3976205539 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f614001a070 con 0x7f61501ab860 2024-09-16T00:45:19.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.382+0000 7f6137fff640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61380a3450 0x7f61380a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:19.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.382+0000 7f6137fff640 1 -- 172.21.15.31:0/3976205539 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f61400ceaf0 con 0x7f61501ab860 2024-09-16T00:45:19.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.387+0000 7f614ed9d640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61380a3450 0x7f61380a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:19.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.388+0000 7f614ed9d640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61380a3450 0x7f61380a5910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f614400b9b0 tx=0x7f614400b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:19.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.408+0000 7f6137fff640 1 -- 172.21.15.31:0/3976205539 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6140020060 con 0x7f61501ab860 2024-09-16T00:45:19.611 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:19 smithi031 ceph-mon[30956]: pgmap v481: 129 pgs: 129 active+clean; 493 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 413 op/s 2024-09-16T00:45:19.895 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:45:19.895 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:45:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.891+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6150109830 con 0x7f61501ab860 2024-09-16T00:45:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.894+0000 7f6137fff640 1 -- 172.21.15.31:0/3976205539 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f614009b510 con 0x7f61501ab860 2024-09-16T00:45:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:45:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.903+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61380a3450 msgr2=0x7f61380a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.903+0000 7f615513b640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61380a3450 0x7f61380a5910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f614400b9b0 tx=0x7f614400b370 comp rx=0 tx=0).stop 2024-09-16T00:45:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.903+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61501ab860 msgr2=0x7f61501abd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.903+0000 7f615513b640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61501ab860 0x7f61501abd10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f614000d720 tx=0x7f614000dbe0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.908+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 shutdown_connections 2024-09-16T00:45:19.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.908+0000 7f615513b640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61501ab860 0x7f61501abd10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.909+0000 7f615513b640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615011b140 0x7f61501a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.909+0000 7f615513b640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f61380a3450 0x7f61380a5910 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.909+0000 7f615513b640 1 --2- 172.21.15.31:0/3976205539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6150109450 0x7f61501a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:19.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.909+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 >> 172.21.15.31:0/3976205539 conn(0x7f615006e140 msgr2=0x7f615011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:19.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.915+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 shutdown_connections 2024-09-16T00:45:19.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:19.916+0000 7f615513b640 1 -- 172.21.15.31:0/3976205539 wait complete. 2024-09-16T00:45:19.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:19 smithi081 ceph-mon[35539]: pgmap v481: 129 pgs: 129 active+clean; 493 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 413 op/s 2024-09-16T00:45:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:19 smithi102 ceph-mon[35364]: pgmap v481: 129 pgs: 129 active+clean; 493 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 413 op/s 2024-09-16T00:45:20.613 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:20 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3976205539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:20.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:20 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3976205539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:21.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:20 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3976205539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:21.308 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0' 2024-09-16T00:45:21.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0 2024-09-16T00:45:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:21.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:21 smithi031 ceph-mon[30956]: pgmap v482: 129 pgs: 129 active+clean; 497 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 409 op/s 2024-09-16T00:45:21.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:21 smithi081 ceph-mon[35539]: pgmap v482: 129 pgs: 129 active+clean; 497 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 409 op/s 2024-09-16T00:45:22.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:21 smithi102 ceph-mon[35364]: pgmap v482: 129 pgs: 129 active+clean; 497 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 409 op/s 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.592+0000 7fb76a607640 1 -- 172.21.15.31:0/888702493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb76411b140 msgr2=0x7fb76411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.592+0000 7fb76a607640 1 --2- 172.21.15.31:0/888702493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb76411b140 0x7fb76411d530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009be0 tx=0x7fb75c02a6f0 comp rx=0 tx=0).stop 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.594+0000 7fb76a607640 1 -- 172.21.15.31:0/888702493 shutdown_connections 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.594+0000 7fb76a607640 1 --2- 172.21.15.31:0/888702493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb76411b140 0x7fb76411d530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.594+0000 7fb76a607640 1 --2- 172.21.15.31:0/888702493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb764109830 0x7fb764109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.594+0000 7fb76a607640 1 --2- 172.21.15.31:0/888702493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb764109450 0x7fb764110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.594+0000 7fb76a607640 1 -- 172.21.15.31:0/888702493 >> 172.21.15.31:0/888702493 conn(0x7fb76406e140 msgr2=0x7fb76406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.595+0000 7fb76a607640 1 -- 172.21.15.31:0/888702493 shutdown_connections 2024-09-16T00:45:22.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.598+0000 7fb76a607640 1 -- 172.21.15.31:0/888702493 wait complete. 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.599+0000 7fb76a607640 1 Processor -- start 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.599+0000 7fb76a607640 1 -- start start 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.599+0000 7fb76a607640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb764109450 0x7fb7641185c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.599+0000 7fb76a607640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb764109830 0x7fb764118b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.599+0000 7fb76a607640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76411b140 0x7fb764111740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.599+0000 7fb76a607640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb764120da0 con 0x7fb76411b140 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.599+0000 7fb76a607640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb764120c20 con 0x7fb764109450 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.599+0000 7fb76a607640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb764120f20 con 0x7fb764109830 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.601+0000 7fb75bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb764109830 0x7fb764118b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.601+0000 7fb768ba4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76411b140 0x7fb764111740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.601+0000 7fb768ba4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76411b140 0x7fb764111740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58546/0 (socket says 172.21.15.31:58546) 2024-09-16T00:45:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.601+0000 7fb768ba4640 1 -- 172.21.15.31:0/482570565 learned_addr learned my addr 172.21.15.31:0/482570565 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.604+0000 7fb763fff640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb764109450 0x7fb7641185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.604+0000 7fb768ba4640 1 -- 172.21.15.31:0/482570565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb764109830 msgr2=0x7fb764118b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.604+0000 7fb768ba4640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb764109830 0x7fb764118b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.604+0000 7fb768ba4640 1 -- 172.21.15.31:0/482570565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb764109450 msgr2=0x7fb7641185c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.604+0000 7fb768ba4640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb764109450 0x7fb7641185c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.604+0000 7fb768ba4640 1 -- 172.21.15.31:0/482570565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb764112000 con 0x7fb76411b140 2024-09-16T00:45:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.605+0000 7fb768ba4640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76411b140 0x7fb764111740 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb75c034040 tx=0x7fb75c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:22.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.606+0000 7fb761ffb640 1 -- 172.21.15.31:0/482570565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c041020 con 0x7fb76411b140 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.606+0000 7fb761ffb640 1 -- 172.21.15.31:0/482570565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75c030880 con 0x7fb76411b140 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.606+0000 7fb761ffb640 1 -- 172.21.15.31:0/482570565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c036040 con 0x7fb76411b140 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.606+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb764112290 con 0x7fb76411b140 2024-09-16T00:45:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.606+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7641162c0 con 0x7fb76411b140 2024-09-16T00:45:22.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.610+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb744005290 con 0x7fb76411b140 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.616+0000 7fb761ffb640 1 -- 172.21.15.31:0/482570565 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb75c044020 con 0x7fb76411b140 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.617+0000 7fb761ffb640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb7500a3450 0x7fb7500a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.617+0000 7fb761ffb640 1 -- 172.21.15.31:0/482570565 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb75c0f0bd0 con 0x7fb76411b140 2024-09-16T00:45:22.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.621+0000 7fb763fff640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb7500a3450 0x7fb7500a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:22.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.624+0000 7fb761ffb640 1 -- 172.21.15.31:0/482570565 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb75c04b070 con 0x7fb76411b140 2024-09-16T00:45:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:22.626+0000 7fb763fff640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb7500a3450 0x7fb7500a5910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb74c005e10 tx=0x7fb74c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:23.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.102+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb744005580 con 0x7fb76411b140 2024-09-16T00:45:23.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.105+0000 7fb761ffb640 1 -- 172.21.15.31:0/482570565 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fb75c0bd560 con 0x7fb76411b140 2024-09-16T00:45:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:45:23.108 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.115+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb7500a3450 msgr2=0x7fb7500a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.115+0000 7fb76a607640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb7500a3450 0x7fb7500a5910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb74c005e10 tx=0x7fb74c005da0 comp rx=0 tx=0).stop 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.115+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76411b140 msgr2=0x7fb764111740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:23.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.115+0000 7fb76a607640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76411b140 0x7fb764111740 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb75c034040 tx=0x7fb75c003c10 comp rx=0 tx=0).stop 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.116+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 shutdown_connections 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.116+0000 7fb76a607640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76411b140 0x7fb764111740 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.116+0000 7fb76a607640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb7500a3450 0x7fb7500a5910 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.116+0000 7fb76a607640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb764109830 0x7fb764118b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.116+0000 7fb76a607640 1 --2- 172.21.15.31:0/482570565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb764109450 0x7fb7641185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.116+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 >> 172.21.15.31:0/482570565 conn(0x7fb76406e140 msgr2=0x7fb76411bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:23.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.117+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 shutdown_connections 2024-09-16T00:45:23.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:23.118+0000 7fb76a607640 1 -- 172.21.15.31:0/482570565 wait complete. 2024-09-16T00:45:23.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:23 smithi031 ceph-mon[30956]: pgmap v483: 129 pgs: 129 active+clean; 502 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 482 op/s 2024-09-16T00:45:23.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:23 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/482570565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:23.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:23 smithi081 ceph-mon[35539]: pgmap v483: 129 pgs: 129 active+clean; 502 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 482 op/s 2024-09-16T00:45:23.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:23 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/482570565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:24.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:23 smithi102 ceph-mon[35364]: pgmap v483: 129 pgs: 129 active+clean; 502 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 482 op/s 2024-09-16T00:45:24.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:23 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/482570565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:24.864 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:24 smithi031 ceph-mon[30956]: pgmap v484: 129 pgs: 129 active+clean; 542 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 738 op/s 2024-09-16T00:45:24.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:45:24.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:24 smithi081 ceph-mon[35539]: pgmap v484: 129 pgs: 129 active+clean; 542 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 738 op/s 2024-09-16T00:45:25.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:24 smithi102 ceph-mon[35364]: pgmap v484: 129 pgs: 129 active+clean; 542 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 738 op/s 2024-09-16T00:45:25.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.129+0000 7fd1876b2640 1 -- 172.21.15.31:0/1074380740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074e30 msgr2=0x7fd18010df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.129+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1074380740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074e30 0x7fd18010df10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd170009900 tx=0x7fd17002a480 comp rx=0 tx=0).stop 2024-09-16T00:45:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.129+0000 7fd1876b2640 1 -- 172.21.15.31:0/1074380740 shutdown_connections 2024-09-16T00:45:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.129+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1074380740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18010e450 0x7fd180110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.129+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1074380740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074e30 0x7fd18010df10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.129+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1074380740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd180074510 0x7fd1800748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.129+0000 7fd1876b2640 1 -- 172.21.15.31:0/1074380740 >> 172.21.15.31:0/1074380740 conn(0x7fd1800fd8f0 msgr2=0x7fd1800ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.129+0000 7fd1876b2640 1 -- 172.21.15.31:0/1074380740 shutdown_connections 2024-09-16T00:45:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.130+0000 7fd1876b2640 1 -- 172.21.15.31:0/1074380740 wait complete. 2024-09-16T00:45:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.130+0000 7fd1876b2640 1 Processor -- start 2024-09-16T00:45:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.130+0000 7fd1876b2640 1 -- start start 2024-09-16T00:45:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd1876b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074510 0x7fd18019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd1876b2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd180074e30 0x7fd18019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd1876b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18010e450 0x7fd18019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd1876b2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd180114170 con 0x7fd18010e450 2024-09-16T00:45:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd1876b2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd180113ff0 con 0x7fd180074510 2024-09-16T00:45:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd1876b2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801142f0 con 0x7fd180074e30 2024-09-16T00:45:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd18544e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074510 0x7fd18019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd185c4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18010e450 0x7fd18019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd184c4d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd180074e30 0x7fd18019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd18544e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074510 0x7fd18019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34434/0 (socket says 172.21.15.31:34434) 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd184c4d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd180074e30 0x7fd18019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:43694/0 (socket says 172.21.15.31:43694) 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd185c4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18010e450 0x7fd18019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58568/0 (socket says 172.21.15.31:58568) 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd18544e640 1 -- 172.21.15.31:0/1399697645 learned_addr learned my addr 172.21.15.31:0/1399697645 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd18544e640 1 -- 172.21.15.31:0/1399697645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd180074e30 msgr2=0x7fd18019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd18544e640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd180074e30 0x7fd18019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd18544e640 1 -- 172.21.15.31:0/1399697645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18010e450 msgr2=0x7fd18019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd18544e640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18010e450 0x7fd18019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd18544e640 1 -- 172.21.15.31:0/1399697645 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18019f6a0 con 0x7fd180074510 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.131+0000 7fd18544e640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074510 0x7fd18019a6f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e9f0 tx=0x7fd17c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.132+0000 7fd1767fc640 1 -- 172.21.15.31:0/1399697645 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00cd80 con 0x7fd180074510 2024-09-16T00:45:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.132+0000 7fd1767fc640 1 -- 172.21.15.31:0/1399697645 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c005500 con 0x7fd180074510 2024-09-16T00:45:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.132+0000 7fd1876b2640 1 -- 172.21.15.31:0/1399697645 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18019f990 con 0x7fd180074510 2024-09-16T00:45:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.132+0000 7fd1767fc640 1 -- 172.21.15.31:0/1399697645 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c010690 con 0x7fd180074510 2024-09-16T00:45:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.132+0000 7fd1876b2640 1 -- 172.21.15.31:0/1399697645 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1801a7270 con 0x7fd180074510 2024-09-16T00:45:25.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.134+0000 7fd189a8b740 1 Processor -- start 2024-09-16T00:45:25.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.134+0000 7fd189a8b740 1 -- start start 2024-09-16T00:45:25.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243043760 0x564243043b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 0x56424304e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x564243050d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd189a8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564242fa92a0 con 0x564243044080 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd189a8b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564242f50b10 con 0x56424304e940 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd189a8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564242fa5260 con 0x564243043760 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd185c4f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243043760 0x564243043b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd184c4d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x564243050d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd185c4f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243043760 0x564243043b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:43708/0 (socket says 172.21.15.31:43708) 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd185c4f640 1 -- 172.21.15.31:0/3313613276 learned_addr learned my addr 172.21.15.31:0/3313613276 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd1767fc640 1 -- 172.21.15.31:0/1399697645 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd17c0046c0 con 0x7fd180074510 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd184c4d640 1 -- 172.21.15.31:0/3313613276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243043760 msgr2=0x564243043b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd184c4d640 1 --2- 172.21.15.31:0/3313613276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243043760 0x564243043b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd184c4d640 1 -- 172.21.15.31:0/3313613276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 msgr2=0x56424304e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd184c4d640 1 --2- 172.21.15.31:0/3313613276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 0x56424304e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd184c4d640 1 -- 172.21.15.31:0/3313613276 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564242d1ab90 con 0x56424304e940 2024-09-16T00:45:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.135+0000 7fd185c4f640 1 --2- 172.21.15.31:0/3313613276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243043760 0x564243043b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.136+0000 7fd184c4d640 1 --2- 172.21.15.31:0/3313613276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x564243050d30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd170005940 tx=0x7fd170005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef6a3d489247db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.136+0000 7fd16b7fe640 1 -- 172.21.15.31:0/3313613276 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170040070 con 0x56424304e940 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.136+0000 7fd16b7fe640 1 -- 172.21.15.31:0/3313613276 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd170031870 con 0x56424304e940 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.136+0000 7fd16b7fe640 1 -- 172.21.15.31:0/3313613276 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170035040 con 0x56424304e940 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.136+0000 7fd1767fc640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd16c0a32c0 0x7fd16c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.136+0000 7fd185c4f640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd16c0a32c0 0x7fd16c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.136+0000 7fd1767fc640 1 -- 172.21.15.31:0/1399697645 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd17c0d0040 con 0x7fd180074510 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 -- 172.21.15.31:0/3313613276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 msgr2=0x564243050d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 --2- 172.21.15.31:0/3313613276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x564243050d30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd170005940 tx=0x7fd170005be0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd185c4f640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd16c0a32c0 0x7fd16c0a5780 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd178009fd0 tx=0x7fd178009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 -- 172.21.15.31:0/3313613276 shutdown_connections 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 --2- 172.21.15.31:0/3313613276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x564243050d30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 --2- 172.21.15.31:0/3313613276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 0x56424304e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 --2- 172.21.15.31:0/3313613276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243043760 0x564243043b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 -- 172.21.15.31:0/3313613276 >> 172.21.15.31:0/3313613276 conn(0x56424303fcf0 msgr2=0x5642430400d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 -- 172.21.15.31:0/3313613276 shutdown_connections 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 -- 172.21.15.31:0/3313613276 wait complete. 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 -- start start 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.137+0000 7fd189a8b740 1 Processor -- start 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243043760 0x56424304b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 0x56424304bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x56424304c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd189a8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564243051510 con 0x564243043760 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd189a8b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564242f316e0 con 0x56424304e940 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd189a8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564243051690 con 0x564243044080 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd189a8b740 10 client.0 _ll_register_callbacks cb 0x564242f55a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd185c4f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x56424304c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd185c4f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x56424304c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34462/0 (socket says 172.21.15.31:34462) 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd185c4f640 1 -- 172.21.15.31:0/1567794104 learned_addr learned my addr 172.21.15.31:0/1567794104 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd18544e640 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243043760 0x56424304b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd184c4d640 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 0x56424304bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd185c4f640 1 -- 172.21.15.31:0/1567794104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 msgr2=0x56424304bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd185c4f640 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 0x56424304bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.138+0000 7fd185c4f640 1 -- 172.21.15.31:0/1567794104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243043760 msgr2=0x56424304b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd185c4f640 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243043760 0x56424304b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd185c4f640 1 -- 172.21.15.31:0/1567794104 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424305aa50 con 0x56424304e940 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd18544e640 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243043760 0x56424304b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd185c4f640 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x56424304c2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd178011dd0 tx=0x7fd178004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554937959e9cefcb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd168ff9640 1 -- 172.21.15.31:0/1567794104 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178004b20 con 0x56424304e940 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd189a8b740 1 -- 172.21.15.31:0/1567794104 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd170031870 con 0x56424304e940 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd168ff9640 1 -- 172.21.15.31:0/1567794104 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd178017030 con 0x56424304e940 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd168ff9640 1 -- 172.21.15.31:0/1567794104 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd178032050 con 0x56424304e940 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd189a8b740 10 client.25508 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd189a8b740 1 -- 172.21.15.31:0/1567794104 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564243065c50 con 0x56424304e940 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.139+0000 7fd168ff9640 1 -- 172.21.15.31:0/1567794104 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd170031870 con 0x56424304e940 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd168ff9640 1 -- 172.21.15.31:0/1567794104 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd178005320 con 0x56424304e940 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd189a8b740 10 client.25508 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd189a8b740 10 client.25508 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:45:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd189a8b740 20 client.25508 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd189a8b740 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564243067580 0x564243069a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd189a8b740 4 client.25508 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd189a8b740 1 -- 172.21.15.31:0/1567794104 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564242fb69d0 con 0x564243067580 2024-09-16T00:45:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd18544e640 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564243067580 0x564243069a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd18544e640 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564243067580 0x564243069a00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379255c672321b2e server_cookie=67c8af3bb1afcc66 in_seq=0 out_seq=0 2024-09-16T00:45:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.140+0000 7fd168ff9640 10 client.25508 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd168ff9640 1 -- 172.21.15.31:0/1567794104 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564242fb69d0 con 0x564243067580 2024-09-16T00:45:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd168ff9640 10 client.25508 handle_command_reply: tid=0 2024-09-16T00:45:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 client.25508 shutdown 2024-09-16T00:45:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 -- 172.21.15.31:0/1567794104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 msgr2=0x56424304c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x56424304c2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd178011dd0 tx=0x7fd178004ab0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 -- 172.21.15.31:0/1567794104 shutdown_connections 2024-09-16T00:45:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56424304e940 0x56424304c2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x564243067580 0x564243069a00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 0x56424304bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 --2- 172.21.15.31:0/1567794104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243043760 0x56424304b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 -- 172.21.15.31:0/1567794104 >> 172.21.15.31:0/1567794104 conn(0x56424303fcf0 msgr2=0x564243045a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd168ff9640 0 client.25508 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.142+0000 7fd189a8b740 1 -- 172.21.15.31:0/1567794104 shutdown_connections 2024-09-16T00:45:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.143+0000 7fd189a8b740 1 -- 172.21.15.31:0/1567794104 wait complete. 2024-09-16T00:45:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.143+0000 7fd189a8b740 20 client.25508 trim_cache size 0 max 16384 2024-09-16T00:45:25.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.161+0000 7fd189a8b740 1 Processor -- start 2024-09-16T00:45:25.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.161+0000 7fd189a8b740 1 -- start start 2024-09-16T00:45:25.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.161+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x56424304d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 0x56424306f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56424304e940 0x56424306f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd189a8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801aab30 con 0x56424304e940 2024-09-16T00:45:25.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd189a8b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801ad3b0 con 0x564243043760 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd189a8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801ab1a0 con 0x564243044080 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd185c4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56424304e940 0x56424306f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd18544e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x56424304d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd18544e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x56424304d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34466/0 (socket says 172.21.15.31:34466) 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd18544e640 1 -- 172.21.15.31:0/4199041211 learned_addr learned my addr 172.21.15.31:0/4199041211 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd184c4d640 1 --2- 172.21.15.31:0/4199041211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 0x56424306f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd18544e640 1 -- 172.21.15.31:0/4199041211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 msgr2=0x56424306f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd18544e640 1 --2- 172.21.15.31:0/4199041211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 0x56424306f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd18544e640 1 -- 172.21.15.31:0/4199041211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56424304e940 msgr2=0x56424306f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd18544e640 1 --2- 172.21.15.31:0/4199041211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56424304e940 0x56424306f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd18544e640 1 -- 172.21.15.31:0/4199041211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424303b260 con 0x564243043760 2024-09-16T00:45:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd185c4f640 1 --2- 172.21.15.31:0/4199041211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56424304e940 0x56424306f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:25.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd18544e640 1 --2- 172.21.15.31:0/4199041211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x56424304d7e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00cfd0 tx=0x7fd17c009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a836e372b6441055 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.162+0000 7fd16affd640 1 -- 172.21.15.31:0/4199041211 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0b5070 con 0x564243043760 2024-09-16T00:45:25.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd16affd640 1 -- 172.21.15.31:0/4199041211 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c09c2b0 con 0x564243043760 2024-09-16T00:45:25.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd16affd640 1 -- 172.21.15.31:0/4199041211 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0abe80 con 0x564243043760 2024-09-16T00:45:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd189a8b740 1 -- 172.21.15.31:0/4199041211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 msgr2=0x56424304d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd189a8b740 1 --2- 172.21.15.31:0/4199041211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x56424304d7e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00cfd0 tx=0x7fd17c009720 comp rx=0 tx=0).stop 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd189a8b740 1 -- 172.21.15.31:0/4199041211 shutdown_connections 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd189a8b740 1 --2- 172.21.15.31:0/4199041211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56424304e940 0x56424306f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd189a8b740 1 --2- 172.21.15.31:0/4199041211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564243044080 0x56424306f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd189a8b740 1 --2- 172.21.15.31:0/4199041211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x56424304d7e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd189a8b740 1 -- 172.21.15.31:0/4199041211 >> 172.21.15.31:0/4199041211 conn(0x56424303fcf0 msgr2=0x56424304d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd189a8b740 1 -- 172.21.15.31:0/4199041211 shutdown_connections 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.163+0000 7fd189a8b740 1 -- 172.21.15.31:0/4199041211 wait complete. 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd189a8b740 1 -- start start 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd189a8b740 1 Processor -- start 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x564243056f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 0x564243048480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd189a8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56424304e940 0x5642430489c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd189a8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801ab870 con 0x564243044080 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd189a8b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801acce0 con 0x564243043760 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd189a8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801abf40 con 0x56424304e940 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd189a8b740 10 client.0 _ll_register_callbacks cb 0x7fd1780114c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.164+0000 7fd184c4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 0x564243048480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd184c4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 0x564243048480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58594/0 (socket says 172.21.15.31:58594) 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd184c4d640 1 -- 172.21.15.31:0/269415176 learned_addr learned my addr 172.21.15.31:0/269415176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd18544e640 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x564243056f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd185c4f640 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56424304e940 0x5642430489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd18544e640 1 -- 172.21.15.31:0/269415176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56424304e940 msgr2=0x5642430489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd18544e640 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56424304e940 0x5642430489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd18544e640 1 -- 172.21.15.31:0/269415176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 msgr2=0x564243048480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd18544e640 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 0x564243048480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd18544e640 1 -- 172.21.15.31:0/269415176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642430490b0 con 0x564243043760 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd18544e640 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x564243056f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0cd7e0 tx=0x7fd17c0a4f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1830a16c60a98954 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd168ff9640 1 -- 172.21.15.31:0/269415176 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0b5040 con 0x564243043760 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd168ff9640 1 -- 172.21.15.31:0/269415176 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd17c0b51e0 con 0x564243043760 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd189a8b740 1 -- 172.21.15.31:0/269415176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd17c09c2b0 con 0x564243043760 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd168ff9640 1 -- 172.21.15.31:0/269415176 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd17c0a4a90 con 0x564243043760 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd189a8b740 10 client.25514 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.165+0000 7fd189a8b740 1 -- 172.21.15.31:0/269415176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5642430d2180 con 0x564243043760 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd168ff9640 1 -- 172.21.15.31:0/269415176 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c09c2b0 con 0x564243043760 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd168ff9640 1 -- 172.21.15.31:0/269415176 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd17c0bc0b0 con 0x564243043760 2024-09-16T00:45:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd189a8b740 10 client.25514 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd189a8b740 10 client.25514 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:45:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd189a8b740 20 client.25514 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd189a8b740 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5642430d2830 0x5642430d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd189a8b740 4 client.25514 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}] 2024-09-16T00:45:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd189a8b740 1 -- 172.21.15.31:0/269415176 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}) -- 0x564242fb0d20 con 0x5642430d2830 2024-09-16T00:45:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd184c4d640 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5642430d2830 0x5642430d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd168ff9640 10 client.25514 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.166+0000 7fd184c4d640 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5642430d2830 0x5642430d4c50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3388c2cbe1376b21 server_cookie=6297a40a18c83644 in_seq=0 out_seq=0 2024-09-16T00:45:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.721+0000 7fd304122640 1 -- 172.21.15.31:0/2289712771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc100a80 msgr2=0x7fd2fc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.721+0000 7fd304122640 1 --2- 172.21.15.31:0/2289712771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc100a80 0x7fd2fc100ee0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009930 tx=0x7fd2ec02a420 comp rx=0 tx=0).stop 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.721+0000 7fd304122640 1 -- 172.21.15.31:0/2289712771 shutdown_connections 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.721+0000 7fd304122640 1 --2- 172.21.15.31:0/2289712771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2fc101420 0x7fd2fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.721+0000 7fd304122640 1 --2- 172.21.15.31:0/2289712771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc100a80 0x7fd2fc100ee0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.721+0000 7fd304122640 1 --2- 172.21.15.31:0/2289712771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2fc106a80 0x7fd2fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.721+0000 7fd304122640 1 -- 172.21.15.31:0/2289712771 >> 172.21.15.31:0/2289712771 conn(0x7fd2fc0fc7a0 msgr2=0x7fd2fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.721+0000 7fd304122640 1 -- 172.21.15.31:0/2289712771 shutdown_connections 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.721+0000 7fd304122640 1 -- 172.21.15.31:0/2289712771 wait complete. 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd304122640 1 Processor -- start 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd304122640 1 -- start start 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd304122640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2fc100a80 0x7fd2fc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd304122640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc101420 0x7fd2fc1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd304122640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd304122640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc110e80 con 0x7fd2fc101420 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd304122640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc110d00 con 0x7fd2fc100a80 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd304122640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc111000 con 0x7fd2fc106a80 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3016bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc101420 0x7fd2fc1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3016bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc101420 0x7fd2fc1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58610/0 (socket says 172.21.15.31:58610) 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3016bd640 1 -- 172.21.15.31:0/3193443104 learned_addr learned my addr 172.21.15.31:0/3193443104 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd301ebe640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2fc100a80 0x7fd2fc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3026bf640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3016bd640 1 -- 172.21.15.31:0/3193443104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2fc106a80 msgr2=0x7fd2fc19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3016bd640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3016bd640 1 -- 172.21.15.31:0/3193443104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2fc100a80 msgr2=0x7fd2fc1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3016bd640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2fc100a80 0x7fd2fc1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3016bd640 1 -- 172.21.15.31:0/3193443104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc19d450 con 0x7fd2fc101420 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd3026bf640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.722+0000 7fd301ebe640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2fc100a80 0x7fd2fc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.723+0000 7fd3016bd640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc101420 0x7fd2fc1989e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009230 tx=0x7fd2ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.723+0000 7fd2f2ffd640 1 -- 172.21.15.31:0/3193443104 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec03f030 con 0x7fd2fc101420 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.723+0000 7fd2f2ffd640 1 -- 172.21.15.31:0/3193443104 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2ec003ad0 con 0x7fd2fc101420 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.723+0000 7fd304122640 1 -- 172.21.15.31:0/3193443104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc19d6e0 con 0x7fd2fc101420 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.723+0000 7fd2f2ffd640 1 -- 172.21.15.31:0/3193443104 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec035070 con 0x7fd2fc101420 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.723+0000 7fd304122640 1 -- 172.21.15.31:0/3193443104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc1a5020 con 0x7fd2fc101420 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.725+0000 7fd2f2ffd640 1 -- 172.21.15.31:0/3193443104 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd2ec042030 con 0x7fd2fc101420 2024-09-16T00:45:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.726+0000 7fd3064fb740 1 Processor -- start 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.726+0000 7fd3064fb740 1 -- start start 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.726+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c14f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.726+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c14f5d0 0x558b3c14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.726+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 0x558b3c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.726+0000 7fd3064fb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558b3c0b52f0 con 0x558b3c150b40 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.726+0000 7fd3064fb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558b3c05ccf0 con 0x558b3c15a370 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.726+0000 7fd3064fb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558b3c0b12b0 con 0x558b3c14f5d0 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd3026bf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c14f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd3026bf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c14f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58616/0 (socket says 172.21.15.31:58616) 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd3026bf640 1 -- 172.21.15.31:0/3445951091 learned_addr learned my addr 172.21.15.31:0/3445951091 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd3016bd640 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 0x558b3c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd2f2ffd640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2e80a3240 0x7fd2e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd301ebe640 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c14f5d0 0x558b3c14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd3026bf640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2e80a3240 0x7fd2e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd301ebe640 1 -- 172.21.15.31:0/3445951091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 msgr2=0x558b3c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd301ebe640 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 0x558b3c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd301ebe640 1 -- 172.21.15.31:0/3445951091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 msgr2=0x558b3c14f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd301ebe640 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c14f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd2f2ffd640 1 -- 172.21.15.31:0/3193443104 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd2ec0f0340 con 0x7fd2fc101420 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd301ebe640 1 -- 172.21.15.31:0/3445951091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3c0413a0 con 0x558b3c14f5d0 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd3026bf640 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c14f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd301ebe640 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c14f5d0 0x558b3c14fa10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009f50 tx=0x7fd2f80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=690a5225250661af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd3026bf640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2e80a3240 0x7fd2e80a5700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4012b20 tx=0x7fd2f4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.727+0000 7fd2e7fff640 1 -- 172.21.15.31:0/3445951091 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8004030 con 0x558b3c14f5d0 2024-09-16T00:45:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd2e7fff640 1 -- 172.21.15.31:0/3445951091 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f8005500 con 0x558b3c14f5d0 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd2e7fff640 1 -- 172.21.15.31:0/3445951091 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8004e10 con 0x558b3c14f5d0 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd3064fb740 1 -- 172.21.15.31:0/3445951091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c14f5d0 msgr2=0x558b3c14fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd3064fb740 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c14f5d0 0x558b3c14fa10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009f50 tx=0x7fd2f80118d0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd3064fb740 1 -- 172.21.15.31:0/3445951091 shutdown_connections 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd3064fb740 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 0x558b3c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd3064fb740 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c14f5d0 0x558b3c14fa10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd3064fb740 1 --2- 172.21.15.31:0/3445951091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c14f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd3064fb740 1 -- 172.21.15.31:0/3445951091 >> 172.21.15.31:0/3445951091 conn(0x558b3c118310 msgr2=0x558b3c14bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd3064fb740 1 -- 172.21.15.31:0/3445951091 shutdown_connections 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.728+0000 7fd3064fb740 1 -- 172.21.15.31:0/3445951091 wait complete. 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3064fb740 1 -- start start 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3064fb740 1 Processor -- start 2024-09-16T00:45:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 0x558b3c156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c15a370 0x558b3c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3064fb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558b3c15d070 con 0x558b3c15a370 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3064fb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558b3c070060 con 0x558b3c14f5d0 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3064fb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558b3c15d1f0 con 0x558b3c150b40 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3064fb740 10 client.0 _ll_register_callbacks cb 0x558b3bdc38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3026bf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c15a370 0x558b3c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3026bf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c15a370 0x558b3c1575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58626/0 (socket says 172.21.15.31:58626) 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3026bf640 1 -- 172.21.15.31:0/2778191248 learned_addr learned my addr 172.21.15.31:0/2778191248 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.729+0000 7fd3016bd640 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 0x558b3c156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd301ebe640 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd3016bd640 1 -- 172.21.15.31:0/2778191248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 msgr2=0x558b3c159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd3016bd640 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd3016bd640 1 -- 172.21.15.31:0/2778191248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c15a370 msgr2=0x558b3c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd3016bd640 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c15a370 0x558b3c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd3016bd640 1 -- 172.21.15.31:0/2778191248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3be263c0 con 0x558b3c150b40 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd3026bf640 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c15a370 0x558b3c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd3016bd640 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 0x558b3c156ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0b89b0 tx=0x7fd2ec043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57ce77eb890d8d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd2e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd2e57fa640 1 -- 172.21.15.31:0/2778191248 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec049040 con 0x558b3c150b40 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd2e57fa640 1 -- 172.21.15.31:0/2778191248 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd2ec0b8bc0 con 0x558b3c150b40 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd3064fb740 1 -- 172.21.15.31:0/2778191248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2f8005500 con 0x558b3c150b40 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.730+0000 7fd2e57fa640 1 -- 172.21.15.31:0/2778191248 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd2ec0d5050 con 0x558b3c150b40 2024-09-16T00:45:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd3064fb740 10 client.25468 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd3064fb740 1 -- 172.21.15.31:0/2778191248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558b3c15d0c0 con 0x558b3c150b40 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd2e57fa640 1 -- 172.21.15.31:0/2778191248 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f8005500 con 0x558b3c150b40 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd2e57fa640 1 -- 172.21.15.31:0/2778191248 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd2ec003c80 con 0x558b3c150b40 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd3064fb740 10 client.25468 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd3064fb740 10 client.25468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd3064fb740 20 client.25468 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd3064fb740 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558b3c169fb0 0x558b3c16c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd3064fb740 4 client.25468 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd3064fb740 1 -- 172.21.15.31:0/2778191248 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b3c0c2860 con 0x558b3c169fb0 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd301ebe640 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558b3c169fb0 0x558b3c16c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd2e57fa640 10 client.25468 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.731+0000 7fd301ebe640 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558b3c169fb0 0x558b3c16c3a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eff2b3d30e30bc41 server_cookie=650d8bbda46e0d2d in_seq=0 out_seq=0 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.733+0000 7fd2e57fa640 1 -- 172.21.15.31:0/2778191248 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b3c0c2860 con 0x558b3c169fb0 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.733+0000 7fd2e57fa640 10 client.25468 handle_command_reply: tid=0 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.733+0000 7fd3064fb740 1 client.25468 shutdown 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 -- 172.21.15.31:0/2778191248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 msgr2=0x558b3c156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 0x558b3c156ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0b89b0 tx=0x7fd2ec043000 comp rx=0 tx=0).stop 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 -- 172.21.15.31:0/2778191248 shutdown_connections 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c15a370 0x558b3c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558b3c169fb0 0x558b3c16c3a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 0x558b3c156ff0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 --2- 172.21.15.31:0/2778191248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd2e57fa640 0 client.25468 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 -- 172.21.15.31:0/2778191248 >> 172.21.15.31:0/2778191248 conn(0x558b3c118310 msgr2=0x558b3c151960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 -- 172.21.15.31:0/2778191248 shutdown_connections 2024-09-16T00:45:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 1 -- 172.21.15.31:0/2778191248 wait complete. 2024-09-16T00:45:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.734+0000 7fd3064fb740 20 client.25468 trim_cache size 0 max 16384 2024-09-16T00:45:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd3064fb740 1 Processor -- start 2024-09-16T00:45:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd3064fb740 1 -- start start 2024-09-16T00:45:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c14f5d0 0x558b3c166310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 0x558b3c150300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 0x558b3c14b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd3064fb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1acc50 con 0x558b3c14f5d0 2024-09-16T00:45:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd3064fb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1af4d0 con 0x558b3c15a370 2024-09-16T00:45:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd3064fb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1ad2c0 con 0x558b3c150b40 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd301ebe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c14f5d0 0x558b3c166310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.747+0000 7fd301ebe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c14f5d0 0x558b3c166310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58640/0 (socket says 172.21.15.31:58640) 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd301ebe640 1 -- 172.21.15.31:0/1091798381 learned_addr learned my addr 172.21.15.31:0/1091798381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd3026bf640 1 --2- 172.21.15.31:0/1091798381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 0x558b3c14b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd3016bd640 1 --2- 172.21.15.31:0/1091798381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 0x558b3c150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd301ebe640 1 -- 172.21.15.31:0/1091798381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 msgr2=0x558b3c150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd301ebe640 1 --2- 172.21.15.31:0/1091798381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 0x558b3c150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd301ebe640 1 -- 172.21.15.31:0/1091798381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 msgr2=0x558b3c14b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd301ebe640 1 --2- 172.21.15.31:0/1091798381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 0x558b3c14b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd301ebe640 1 -- 172.21.15.31:0/1091798381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3c15ca90 con 0x558b3c14f5d0 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd301ebe640 1 --2- 172.21.15.31:0/1091798381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c14f5d0 0x558b3c166310 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009ef0 tx=0x7fd2f8004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5fcefafa9d06c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd2e77fe640 1 -- 172.21.15.31:0/1091798381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8004050 con 0x558b3c14f5d0 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd2e77fe640 1 -- 172.21.15.31:0/1091798381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f80041f0 con 0x558b3c14f5d0 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.748+0000 7fd2e77fe640 1 -- 172.21.15.31:0/1091798381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f800f7a0 con 0x558b3c14f5d0 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.749+0000 7fd3064fb740 1 -- 172.21.15.31:0/1091798381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c14f5d0 msgr2=0x558b3c166310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.749+0000 7fd3064fb740 1 --2- 172.21.15.31:0/1091798381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c14f5d0 0x558b3c166310 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009ef0 tx=0x7fd2f8004970 comp rx=0 tx=0).stop 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.749+0000 7fd3064fb740 1 -- 172.21.15.31:0/1091798381 shutdown_connections 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.749+0000 7fd3064fb740 1 --2- 172.21.15.31:0/1091798381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c15a370 0x558b3c14b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.749+0000 7fd3064fb740 1 --2- 172.21.15.31:0/1091798381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c150b40 0x558b3c150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.749+0000 7fd3064fb740 1 --2- 172.21.15.31:0/1091798381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c14f5d0 0x558b3c166310 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.749+0000 7fd3064fb740 1 -- 172.21.15.31:0/1091798381 >> 172.21.15.31:0/1091798381 conn(0x558b3c118310 msgr2=0x558b3c146a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.749+0000 7fd3064fb740 1 -- 172.21.15.31:0/1091798381 shutdown_connections 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.749+0000 7fd3064fb740 1 -- 172.21.15.31:0/1091798381 wait complete. 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.750+0000 7fd3064fb740 1 -- start start 2024-09-16T00:45:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.750+0000 7fd3064fb740 1 Processor -- start 2024-09-16T00:45:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.750+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c162b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.750+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c153d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3064fb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c15a370 0x558b3c154270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3064fb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1ad990 con 0x558b3c150b40 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3064fb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1aee00 con 0x558b3c14f5d0 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3064fb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1ae060 con 0x558b3c15a370 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3064fb740 10 client.0 _ll_register_callbacks cb 0x7fd2ec009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd301ebe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c162b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3016bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd301ebe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c162b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34534/0 (socket says 172.21.15.31:34534) 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd301ebe640 1 -- 172.21.15.31:0/4158194062 learned_addr learned my addr 172.21.15.31:0/4158194062 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3026bf640 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c15a370 0x558b3c154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3016bd640 1 -- 172.21.15.31:0/4158194062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c15a370 msgr2=0x558b3c154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3016bd640 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c15a370 0x558b3c154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3016bd640 1 -- 172.21.15.31:0/4158194062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 msgr2=0x558b3c162b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3016bd640 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c162b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3016bd640 1 -- 172.21.15.31:0/4158194062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3c154a80 con 0x558b3c150b40 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd301ebe640 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c162b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd3016bd640 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c153d30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0b8f20 tx=0x7fd2ec005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe24089866d45c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd2e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.751+0000 7fd2e57fa640 1 -- 172.21.15.31:0/4158194062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec049040 con 0x558b3c150b40 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd2e57fa640 1 -- 172.21.15.31:0/4158194062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd2ec0c4d30 con 0x558b3c150b40 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd3064fb740 1 -- 172.21.15.31:0/4158194062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2f80041f0 con 0x558b3c150b40 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd2e57fa640 1 -- 172.21.15.31:0/4158194062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd2ec0a4050 con 0x558b3c150b40 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd3064fb740 10 client.15861 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd3064fb740 1 -- 172.21.15.31:0/4158194062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558b3c1d9060 con 0x558b3c150b40 2024-09-16T00:45:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd2e57fa640 1 -- 172.21.15.31:0/4158194062 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f80041f0 con 0x558b3c150b40 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd2e57fa640 1 -- 172.21.15.31:0/4158194062 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd2ec0bb9a0 con 0x558b3c150b40 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd3064fb740 10 client.15861 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd3064fb740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd3064fb740 20 client.15861 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd3064fb740 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558b3c1d9650 0x558b3c1dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd3064fb740 4 client.15861 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd3064fb740 1 -- 172.21.15.31:0/4158194062 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b3c0bcd70 con 0x558b3c1d9650 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.752+0000 7fd301ebe640 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558b3c1d9650 0x558b3c1dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.753+0000 7fd2e57fa640 10 client.15861 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.753+0000 7fd301ebe640 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558b3c1d9650 0x558b3c1dba70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=783c7dd2b5e11b92 server_cookie=74fba06093cf7da2 in_seq=0 out_seq=0 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.754+0000 7fd2e57fa640 1 -- 172.21.15.31:0/4158194062 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b3c0bcd70 con 0x558b3c1d9650 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.754+0000 7fd2e57fa640 10 client.15861 handle_command_reply: tid=0 2024-09-16T00:45:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.754+0000 7fd3064fb740 1 client.15861 shutdown 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 -- 172.21.15.31:0/4158194062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 msgr2=0x558b3c153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c153d30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0b8f20 tx=0x7fd2ec005b50 comp rx=0 tx=0).stop 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 -- 172.21.15.31:0/4158194062 shutdown_connections 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558b3c1d9650 0x558b3c1dba70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b3c15a370 0x558b3c154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd2e57fa640 0 client.15861 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b3c150b40 0x558b3c153d30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 --2- 172.21.15.31:0/4158194062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b3c14f5d0 0x558b3c162b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 -- 172.21.15.31:0/4158194062 >> 172.21.15.31:0/4158194062 conn(0x558b3c118310 msgr2=0x558b3c14b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 -- 172.21.15.31:0/4158194062 shutdown_connections 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 1 -- 172.21.15.31:0/4158194062 wait complete. 2024-09-16T00:45:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.755+0000 7fd3064fb740 20 client.15861 trim_cache size 0 max 16384 2024-09-16T00:45:25.760 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:45:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 -- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2e80a3240 msgr2=0x7fd2e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2e80a3240 0x7fd2e80a5700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4012b20 tx=0x7fd2f4004ab0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 -- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc101420 msgr2=0x7fd2fc1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc101420 0x7fd2fc1989e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009230 tx=0x7fd2ec004270 comp rx=0 tx=0).stop 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 -- 172.21.15.31:0/3193443104 shutdown_connections 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2e80a3240 0x7fd2e80a5700 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2fc101420 0x7fd2fc1989e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 --2- 172.21.15.31:0/3193443104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2fc100a80 0x7fd2fc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 -- 172.21.15.31:0/3193443104 >> 172.21.15.31:0/3193443104 conn(0x7fd2fc0fc7a0 msgr2=0x7fd2fc06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 -- 172.21.15.31:0/3193443104 shutdown_connections 2024-09-16T00:45:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:25.756+0000 7fd304122640 1 -- 172.21.15.31:0/3193443104 wait complete. 2024-09-16T00:45:26.188 INFO:tasks.workunit.client.0.smithi031.stdout: 95 989786 193 629433 249 598176 798 2024-09-16T00:45:27.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:26 smithi031 ceph-mon[30956]: pgmap v485: 129 pgs: 129 active+clean; 554 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.0 MiB/s wr, 771 op/s 2024-09-16T00:45:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:26 smithi081 ceph-mon[35539]: pgmap v485: 129 pgs: 129 active+clean; 554 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.0 MiB/s wr, 771 op/s 2024-09-16T00:45:27.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:26 smithi102 ceph-mon[35364]: pgmap v485: 129 pgs: 129 active+clean; 554 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.0 MiB/s wr, 771 op/s 2024-09-16T00:45:27.446 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:45:27.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:45:27.729 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:28.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.221+0000 7fd2a3858640 1 -- 172.21.15.31:0/306055006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 msgr2=0x7fd29c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:28.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.222+0000 7fd2a3858640 1 --2- 172.21.15.31:0/306055006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 0x7fd29c10d130 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd294009ca0 tx=0x7fd29402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.222+0000 7fd2a3858640 1 -- 172.21.15.31:0/306055006 shutdown_connections 2024-09-16T00:45:28.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.222+0000 7fd2a3858640 1 --2- 172.21.15.31:0/306055006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 0x7fd29c10d130 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.222+0000 7fd2a3858640 1 --2- 172.21.15.31:0/306055006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd29c100e30 0x7fd29c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.222+0000 7fd2a3858640 1 --2- 172.21.15.31:0/306055006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd29c100480 0x7fd29c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.222+0000 7fd2a3858640 1 -- 172.21.15.31:0/306055006 >> 172.21.15.31:0/306055006 conn(0x7fd29c077210 msgr2=0x7fd29c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:28.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.222+0000 7fd2a3858640 1 -- 172.21.15.31:0/306055006 shutdown_connections 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.222+0000 7fd2a3858640 1 -- 172.21.15.31:0/306055006 wait complete. 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.222+0000 7fd2a3858640 1 Processor -- start 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a3858640 1 -- start start 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a3858640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd29c100480 0x7fd29c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a3858640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd29c100e30 0x7fd29c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a3858640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 0x7fd29c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a3858640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c1105c0 con 0x7fd29c1055b0 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a3858640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c110440 con 0x7fd29c100480 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a3858640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c110740 con 0x7fd29c100e30 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a1df5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 0x7fd29c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a15f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd29c100480 0x7fd29c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a15f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd29c100480 0x7fd29c197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:52844/0 (socket says 172.21.15.31:52844) 2024-09-16T00:45:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a1df5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 0x7fd29c19bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33406/0 (socket says 172.21.15.31:33406) 2024-09-16T00:45:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a15f4640 1 -- 172.21.15.31:0/1899236464 learned_addr learned my addr 172.21.15.31:0/1899236464 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd29c100e30 0x7fd29c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a1df5640 1 -- 172.21.15.31:0/1899236464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd29c100e30 msgr2=0x7fd29c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a1df5640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd29c100e30 0x7fd29c197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a1df5640 1 -- 172.21.15.31:0/1899236464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd29c100480 msgr2=0x7fd29c197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a1df5640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd29c100480 0x7fd29c197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a1df5640 1 -- 172.21.15.31:0/1899236464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29c19c2d0 con 0x7fd29c1055b0 2024-09-16T00:45:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.223+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd29c100480 0x7fd29c197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.224+0000 7fd2a1df5640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 0x7fd29c19bbf0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd29402ade0 tx=0x7fd2940319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:28.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.224+0000 7fd28e7fc640 1 -- 172.21.15.31:0/1899236464 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294003b70 con 0x7fd29c1055b0 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.224+0000 7fd2a3858640 1 -- 172.21.15.31:0/1899236464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29c19c560 con 0x7fd29c1055b0 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.224+0000 7fd28e7fc640 1 -- 172.21.15.31:0/1899236464 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd294003d10 con 0x7fd29c1055b0 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.224+0000 7fd28e7fc640 1 -- 172.21.15.31:0/1899236464 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29402fba0 con 0x7fd29c1055b0 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.224+0000 7fd2a3858640 1 -- 172.21.15.31:0/1899236464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29c1a3ea0 con 0x7fd29c1055b0 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.226+0000 7fd28e7fc640 1 -- 172.21.15.31:0/1899236464 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd2940046c0 con 0x7fd29c1055b0 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a5c31740 1 Processor -- start 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a5c31740 1 -- start start 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 0x557e202b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b5320 0x557e202c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202c1f50 0x557e202c4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557e2021a280 con 0x557e202c1f50 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557e201c1b10 con 0x557e202b5320 2024-09-16T00:45:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557e20216240 con 0x557e202b4a00 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a15f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b5320 0x557e202c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a0df3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202c1f50 0x557e202c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a15f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b5320 0x557e202c1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:52850/0 (socket says 172.21.15.31:52850) 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a0df3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202c1f50 0x557e202c4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33414/0 (socket says 172.21.15.31:33414) 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a15f4640 1 -- 172.21.15.31:0/1604015131 learned_addr learned my addr 172.21.15.31:0/1604015131 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd28e7fc640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2880a3240 0x7fd2880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.227+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1604015131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 0x557e202b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2a1df5640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2880a3240 0x7fd2880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd28e7fc640 1 -- 172.21.15.31:0/1899236464 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd2940f0c50 con 0x7fd29c1055b0 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2a15f4640 1 -- 172.21.15.31:0/1604015131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 msgr2=0x557e202b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1604015131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 0x557e202b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2a15f4640 1 -- 172.21.15.31:0/1604015131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202c1f50 msgr2=0x557e202c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1604015131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202c1f50 0x557e202c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2a15f4640 1 -- 172.21.15.31:0/1604015131 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e201ce1f0 con 0x557e202b5320 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1604015131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202c1f50 0x557e202c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1604015131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b5320 0x557e202c1a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd298011650 tx=0x7fd298009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f7d2ed10926da24 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2a1df5640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2880a3240 0x7fd2880a5700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd2940b9320 tx=0x7fd294087850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2877fe640 1 -- 172.21.15.31:0/1604015131 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29801e070 con 0x557e202b5320 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2877fe640 1 -- 172.21.15.31:0/1604015131 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29800fba0 con 0x557e202b5320 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.228+0000 7fd2877fe640 1 -- 172.21.15.31:0/1604015131 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298019cd0 con 0x557e202b5320 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1604015131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b5320 msgr2=0x557e202c1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1604015131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b5320 0x557e202c1a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd298011650 tx=0x7fd298009f90 comp rx=0 tx=0).stop 2024-09-16T00:45:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1604015131 shutdown_connections 2024-09-16T00:45:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1604015131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202c1f50 0x557e202c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1604015131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b5320 0x557e202c1a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1604015131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 0x557e202b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1604015131 >> 172.21.15.31:0/1604015131 conn(0x557e202b10f0 msgr2=0x557e202b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1604015131 shutdown_connections 2024-09-16T00:45:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1604015131 wait complete. 2024-09-16T00:45:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.229+0000 7fd2a5c31740 1 -- start start 2024-09-16T00:45:28.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a5c31740 1 Processor -- start 2024-09-16T00:45:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b4a00 0x557e202bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202c1f50 0x557e202cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557e202b7a40 con 0x557e202b5320 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557e201d4f10 con 0x557e202b4a00 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557e202b7bc0 con 0x557e202c1f50 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a5c31740 10 client.0 _ll_register_callbacks cb 0x557e2019da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a15f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a0df3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b4a00 0x557e202bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a15f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202bdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33424/0 (socket says 172.21.15.31:33424) 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a15f4640 1 -- 172.21.15.31:0/3335696850 learned_addr learned my addr 172.21.15.31:0/3335696850 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202c1f50 0x557e202cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a15f4640 1 -- 172.21.15.31:0/3335696850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202c1f50 msgr2=0x557e202cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202c1f50 0x557e202cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a15f4640 1 -- 172.21.15.31:0/3335696850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b4a00 msgr2=0x557e202bd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.230+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b4a00 0x557e202bd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd2a15f4640 1 -- 172.21.15.31:0/3335696850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e1ff8b6a0 con 0x557e202b5320 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202bdac0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd298019810 tx=0x7fd298009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a139c5f32b18583 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd284ff9640 1 -- 172.21.15.31:0/3335696850 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29801e040 con 0x557e202b5320 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd284ff9640 1 -- 172.21.15.31:0/3335696850 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd29800fd50 con 0x557e202b5320 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd2a5c31740 1 -- 172.21.15.31:0/3335696850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29800fba0 con 0x557e202b5320 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd284ff9640 1 -- 172.21.15.31:0/3335696850 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd29802f050 con 0x557e202b5320 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd2a5c31740 10 client.15873 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd2a5c31740 1 -- 172.21.15.31:0/3335696850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557e202d29d0 con 0x557e202b5320 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.231+0000 7fd284ff9640 1 -- 172.21.15.31:0/3335696850 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29800fba0 con 0x557e202b5320 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd284ff9640 1 -- 172.21.15.31:0/3335696850 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd29802a030 con 0x557e202b5320 2024-09-16T00:45:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd2a5c31740 10 client.15873 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:28.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd2a5c31740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:28.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd2a5c31740 20 client.15873 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:28.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557e202d3dd0 0x557e202d61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd2a5c31740 4 client.15873 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:28.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd2a5c31740 1 -- 172.21.15.31:0/3335696850 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e202277f0 con 0x557e202d3dd0 2024-09-16T00:45:28.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557e202d3dd0 0x557e202d61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd284ff9640 10 client.15873 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:28.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.232+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557e202d3dd0 0x557e202d61c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff05163551f2c54 server_cookie=4b45484a3608247 in_seq=0 out_seq=0 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd284ff9640 1 -- 172.21.15.31:0/3335696850 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e202277f0 con 0x557e202d3dd0 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd284ff9640 10 client.15873 handle_command_reply: tid=0 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 client.15873 shutdown 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 -- 172.21.15.31:0/3335696850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 msgr2=0x557e202bdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202bdac0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd298019810 tx=0x7fd298009ed0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 -- 172.21.15.31:0/3335696850 shutdown_connections 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557e202d3dd0 0x557e202d61c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202c1f50 0x557e202cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202bdac0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd284ff9640 0 client.15873 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/3335696850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202b4a00 0x557e202bd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 -- 172.21.15.31:0/3335696850 >> 172.21.15.31:0/3335696850 conn(0x557e202b10f0 msgr2=0x557e202b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 -- 172.21.15.31:0/3335696850 shutdown_connections 2024-09-16T00:45:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 1 -- 172.21.15.31:0/3335696850 wait complete. 2024-09-16T00:45:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.267+0000 7fd2a5c31740 20 client.15873 trim_cache size 0 max 16384 2024-09-16T00:45:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.286+0000 7fd2a5c31740 1 Processor -- start 2024-09-16T00:45:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.286+0000 7fd2a5c31740 1 -- start start 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.286+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 0x557e202b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202db980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 0x557e202dbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c1aaab0 con 0x557e202b5320 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c1ad330 con 0x557e202c1f50 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c1ab120 con 0x557e202b4a00 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a15f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a15f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202db980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33436/0 (socket says 172.21.15.31:33436) 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a15f4640 1 -- 172.21.15.31:0/1144939931 learned_addr learned my addr 172.21.15.31:0/1144939931 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 0x557e202dbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 0x557e202b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a15f4640 1 -- 172.21.15.31:0/1144939931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 msgr2=0x557e202b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 0x557e202b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a15f4640 1 -- 172.21.15.31:0/1144939931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 msgr2=0x557e202dbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 0x557e202dbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a15f4640 1 -- 172.21.15.31:0/1144939931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e202b5150 con 0x557e202b5320 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 0x557e202b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202db980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd2980115a0 tx=0x7fd298005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe002b94825d6d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd286ffd640 1 -- 172.21.15.31:0/1144939931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298015020 con 0x557e202b5320 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd286ffd640 1 -- 172.21.15.31:0/1144939931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd298019970 con 0x557e202b5320 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.287+0000 7fd286ffd640 1 -- 172.21.15.31:0/1144939931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29801e070 con 0x557e202b5320 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.288+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1144939931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 msgr2=0x557e202db980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.288+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202db980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd2980115a0 tx=0x7fd298005780 comp rx=0 tx=0).stop 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.288+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1144939931 shutdown_connections 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.288+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 0x557e202dbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.288+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b5320 0x557e202db980 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.288+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1144939931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b4a00 0x557e202b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.288+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1144939931 >> 172.21.15.31:0/1144939931 conn(0x557e202b10f0 msgr2=0x557e202b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:28.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.288+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1144939931 shutdown_connections 2024-09-16T00:45:28.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.288+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1144939931 wait complete. 2024-09-16T00:45:28.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.289+0000 7fd2a5c31740 1 -- start start 2024-09-16T00:45:28.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.289+0000 7fd2a5c31740 1 Processor -- start 2024-09-16T00:45:28.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.289+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b4a00 0x557e202cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.289+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b5320 0x557e202cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.289+0000 7fd2a5c31740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 0x557e2033d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b4a00 0x557e202cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b4a00 0x557e202cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33444/0 (socket says 172.21.15.31:33444) 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 -- 172.21.15.31:0/1983377640 learned_addr learned my addr 172.21.15.31:0/1983377640 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a5c31740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c1ab7f0 con 0x557e202b4a00 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c1acc60 con 0x557e202c1f50 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c1abec0 con 0x557e202b5320 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b5320 0x557e202cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 0x557e2033d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a5c31740 10 client.0 _ll_register_callbacks cb 0x7fd29801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 -- 172.21.15.31:0/1983377640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b5320 msgr2=0x557e202cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b5320 0x557e202cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 -- 172.21.15.31:0/1983377640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 msgr2=0x557e2033d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 0x557e2033d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 -- 172.21.15.31:0/1983377640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e202cc2d0 con 0x557e202b4a00 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a0df3640 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b4a00 0x557e202cb490 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x557e202b21e0 tx=0x7fd29800fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716b7e654ac2366d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd284ff9640 1 -- 172.21.15.31:0/1983377640 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290011e60 con 0x557e202b4a00 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd298019970 con 0x557e202b4a00 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.290+0000 7fd284ff9640 1 -- 172.21.15.31:0/1983377640 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd290026e80 con 0x557e202b4a00 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd284ff9640 1 -- 172.21.15.31:0/1983377640 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd29000fda0 con 0x557e202b4a00 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd2a5c31740 10 client.15879 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557e2033f060 con 0x557e202b4a00 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd284ff9640 1 -- 172.21.15.31:0/1983377640 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd298019970 con 0x557e202b4a00 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd284ff9640 1 -- 172.21.15.31:0/1983377640 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd290024070 con 0x557e202b4a00 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd2a5c31740 10 client.15879 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd2a5c31740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd2a5c31740 20 client.15879 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557e2033f780 0x557e20341b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd2a5c31740 4 client.15879 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:45:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557e202b8180 con 0x557e2033f780 2024-09-16T00:45:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.291+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557e2033f780 0x557e20341b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.292+0000 7fd2a15f4640 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557e2033f780 0x557e20341b90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7cd7fc2a9f1ee9 server_cookie=3446004a6e62edf4 in_seq=0 out_seq=0 2024-09-16T00:45:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.292+0000 7fd284ff9640 10 client.15879 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:28.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.510+0000 7fd284ff9640 1 -- 172.21.15.31:0/1983377640 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557e202b8180 con 0x557e2033f780 2024-09-16T00:45:28.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.510+0000 7fd284ff9640 10 client.15879 handle_command_reply: tid=0 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.510+0000 7fd2a5c31740 1 client.15879 shutdown 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b4a00 msgr2=0x557e202cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b4a00 0x557e202cb490 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x557e202b21e0 tx=0x7fd29800fa60 comp rx=0 tx=0).stop 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 shutdown_connections 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e202c1f50 0x557e2033d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557e2033f780 0x557e20341b90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557e202b5320 0x557e202cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 --2- 172.21.15.31:0/1983377640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e202b4a00 0x557e202cb490 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 >> 172.21.15.31:0/1983377640 conn(0x557e202b10f0 msgr2=0x557e202c1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd284ff9640 0 client.15879 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 shutdown_connections 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 1 -- 172.21.15.31:0/1983377640 wait complete. 2024-09-16T00:45:28.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.511+0000 7fd2a5c31740 20 client.15879 trim_cache size 0 max 16384 2024-09-16T00:45:28.513 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:45:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:45:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "4be73a9d-ac52-4ff9-86f0-07ecc97fcfc9", 2024-09-16T00:45:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:45:28.513 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 -- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2880a3240 msgr2=0x7fd2880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2880a3240 0x7fd2880a5700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd2940b9320 tx=0x7fd294087850 comp rx=0 tx=0).stop 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 -- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 msgr2=0x7fd29c19bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 0x7fd29c19bbf0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd29402ade0 tx=0x7fd2940319f0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 -- 172.21.15.31:0/1899236464 shutdown_connections 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd29c1055b0 0x7fd29c19bbf0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd2880a3240 0x7fd2880a5700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd29c100e30 0x7fd29c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 --2- 172.21.15.31:0/1899236464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd29c100480 0x7fd29c197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 -- 172.21.15.31:0/1899236464 >> 172.21.15.31:0/1899236464 conn(0x7fd29c077210 msgr2=0x7fd29c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 -- 172.21.15.31:0/1899236464 shutdown_connections 2024-09-16T00:45:28.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:28.512+0000 7fd2a3858640 1 -- 172.21.15.31:0/1899236464 wait complete. 2024-09-16T00:45:29.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:28 smithi031 ceph-mon[30956]: pgmap v486: 129 pgs: 129 active+clean; 559 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.7 MiB/s wr, 994 op/s 2024-09-16T00:45:29.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:28 smithi081 ceph-mon[35539]: pgmap v486: 129 pgs: 129 active+clean; 559 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.7 MiB/s wr, 994 op/s 2024-09-16T00:45:29.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:28 smithi102 ceph-mon[35364]: pgmap v486: 129 pgs: 129 active+clean; 559 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.7 MiB/s wr, 994 op/s 2024-09-16T00:45:30.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:29 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:45:30.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:29 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:45:30.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:29 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:45:30.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:29 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:45:30.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:29 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:45:30.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:29 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:45:30.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:29 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:45:30.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:29 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:45:30.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:29 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:45:31.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:30 smithi031 ceph-mon[30956]: pgmap v487: 129 pgs: 129 active+clean; 557 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.9 MiB/s wr, 952 op/s 2024-09-16T00:45:31.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:30 smithi081 ceph-mon[35539]: pgmap v487: 129 pgs: 129 active+clean; 557 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.9 MiB/s wr, 952 op/s 2024-09-16T00:45:31.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:30 smithi102 ceph-mon[35364]: pgmap v487: 129 pgs: 129 active+clean; 557 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.9 MiB/s wr, 952 op/s 2024-09-16T00:45:31.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:45:32.099 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.619+0000 7f13cf085640 1 -- 172.21.15.31:0/3155342655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 msgr2=0x7f13c810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.619+0000 7f13cf085640 1 --2- 172.21.15.31:0/3155342655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 0x7f13c810e610 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009ca0 tx=0x7f13c002a950 comp rx=0 tx=0).stop 2024-09-16T00:45:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 -- 172.21.15.31:0/3155342655 shutdown_connections 2024-09-16T00:45:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 --2- 172.21.15.31:0/3155342655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 0x7f13c810e610 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 --2- 172.21.15.31:0/3155342655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13c8103660 0x7f13c810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 --2- 172.21.15.31:0/3155342655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c8102d40 0x7f13c8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 -- 172.21.15.31:0/3155342655 >> 172.21.15.31:0/3155342655 conn(0x7f13c80fc7a0 msgr2=0x7f13c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 -- 172.21.15.31:0/3155342655 shutdown_connections 2024-09-16T00:45:32.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 -- 172.21.15.31:0/3155342655 wait complete. 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 Processor -- start 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 -- start start 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13c8102d40 0x7f13c819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c8103660 0x7f13c819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.620+0000 7f13cf085640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 0x7f13c819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cf085640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8111f70 con 0x7f13c810c170 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cf085640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8111df0 con 0x7f13c8103660 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cf085640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81120f0 con 0x7f13c8102d40 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cd622640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 0x7f13c819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cce21640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13c8102d40 0x7f13c819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c8103660 0x7f13c819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cd622640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 0x7f13c819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33460/0 (socket says 172.21.15.31:33460) 2024-09-16T00:45:32.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cd622640 1 -- 172.21.15.31:0/2052556137 learned_addr learned my addr 172.21.15.31:0/2052556137 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cce21640 1 -- 172.21.15.31:0/2052556137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c8103660 msgr2=0x7f13c819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cce21640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c8103660 0x7f13c819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cce21640 1 -- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 msgr2=0x7f13c819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cce21640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 0x7f13c819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cce21640 1 -- 172.21.15.31:0/2052556137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c819f580 con 0x7f13c8102d40 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cd622640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 0x7f13c819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cce21640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13c8102d40 0x7f13c819a5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00e9f0 tx=0x7f13bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13c5ffb640 1 -- 172.21.15.31:0/2052556137 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00ccc0 con 0x7f13c8102d40 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13c5ffb640 1 -- 172.21.15.31:0/2052556137 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13bc010910 con 0x7f13c8102d40 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13cf085640 1 -- 172.21.15.31:0/2052556137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c819f870 con 0x7f13c8102d40 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.621+0000 7f13c5ffb640 1 -- 172.21.15.31:0/2052556137 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc018960 con 0x7f13c8102d40 2024-09-16T00:45:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.622+0000 7f13cf085640 1 -- 172.21.15.31:0/2052556137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81a7150 con 0x7f13c8102d40 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13d145e740 1 Processor -- start 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13d145e740 1 -- start start 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13c5ffb640 1 -- 172.21.15.31:0/2052556137 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f13bc010ac0 con 0x7f13c8102d40 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13d145e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a870064a0 0x556a87006880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13d145e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 0x556a87011140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13d145e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87011680 0x556a87013a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13d145e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556a86f6c2e0 con 0x556a870064a0 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13d145e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556a86f13ce0 con 0x556a87011680 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13d145e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556a86f682a0 con 0x556a87006dc0 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13cd622640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a870064a0 0x556a87006880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13cce21640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 0x556a87011140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13cd622640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a870064a0 0x556a87006880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33468/0 (socket says 172.21.15.31:33468) 2024-09-16T00:45:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13cd622640 1 -- 172.21.15.31:0/495967208 learned_addr learned my addr 172.21.15.31:0/495967208 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.625+0000 7f13c7fff640 1 --2- 172.21.15.31:0/495967208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87011680 0x556a87013a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13cd622640 1 -- 172.21.15.31:0/495967208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 msgr2=0x556a87011140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13cd622640 1 --2- 172.21.15.31:0/495967208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 0x556a87011140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13cd622640 1 -- 172.21.15.31:0/495967208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87011680 msgr2=0x556a87013a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13cd622640 1 --2- 172.21.15.31:0/495967208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87011680 0x556a87013a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13cd622640 1 -- 172.21.15.31:0/495967208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a86ef8440 con 0x556a870064a0 2024-09-16T00:45:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13cd622640 1 --2- 172.21.15.31:0/495967208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a870064a0 0x556a87006880 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009400 tx=0x7f13c002fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f06aab8615faed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13b2ffd640 1 -- 172.21.15.31:0/495967208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0048070 con 0x556a870064a0 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13c5ffb640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13b80a3240 0x7f13b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13b2ffd640 1 -- 172.21.15.31:0/495967208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c0003970 con 0x556a870064a0 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13b2ffd640 1 -- 172.21.15.31:0/495967208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0034040 con 0x556a870064a0 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.626+0000 7f13c5ffb640 1 -- 172.21.15.31:0/2052556137 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f13bc0d0050 con 0x7f13c8102d40 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13cce21640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13b80a3240 0x7f13b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13d145e740 1 -- 172.21.15.31:0/495967208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a870064a0 msgr2=0x556a87006880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13cce21640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13b80a3240 0x7f13b80a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0cdb10 tx=0x7f13bc0d3b30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13d145e740 1 --2- 172.21.15.31:0/495967208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a870064a0 0x556a87006880 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009400 tx=0x7f13c002fe10 comp rx=0 tx=0).stop 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13d145e740 1 -- 172.21.15.31:0/495967208 shutdown_connections 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13d145e740 1 --2- 172.21.15.31:0/495967208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87011680 0x556a87013a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13d145e740 1 --2- 172.21.15.31:0/495967208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 0x556a87011140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13d145e740 1 --2- 172.21.15.31:0/495967208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a870064a0 0x556a87006880 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13d145e740 1 -- 172.21.15.31:0/495967208 >> 172.21.15.31:0/495967208 conn(0x556a87002a30 msgr2=0x556a87002e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13d145e740 1 -- 172.21.15.31:0/495967208 shutdown_connections 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.627+0000 7f13d145e740 1 -- 172.21.15.31:0/495967208 wait complete. 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13d145e740 1 -- start start 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13d145e740 1 Processor -- start 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13d145e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87010fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13d145e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 0x556a8700e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13d145e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87011680 0x556a8700e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13d145e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556a87014250 con 0x556a87011680 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13d145e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556a86f27050 con 0x556a870064a0 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13d145e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556a870143d0 con 0x556a87006dc0 2024-09-16T00:45:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13d145e740 10 client.0 _ll_register_callbacks cb 0x556a86f67160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13cd622640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87011680 0x556a8700e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 0x556a8700e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13cd622640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87010fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13cd622640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87010fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:52918/0 (socket says 172.21.15.31:52918) 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 0x556a8700e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:36920/0 (socket says 172.21.15.31:36920) 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.628+0000 7f13cd622640 1 -- 172.21.15.31:0/3098728378 learned_addr learned my addr 172.21.15.31:0/3098728378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13cd622640 1 -- 172.21.15.31:0/3098728378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 msgr2=0x556a8700e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13cd622640 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 0x556a8700e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13cd622640 1 -- 172.21.15.31:0/3098728378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87011680 msgr2=0x556a8700e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13cd622640 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87011680 0x556a8700e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13cd622640 1 -- 172.21.15.31:0/3098728378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a86cdd3c0 con 0x556a870064a0 2024-09-16T00:45:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13cd622640 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87010fc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f13c002ae70 tx=0x7f13c002a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8fd445ff3359e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13a7fff640 1 -- 172.21.15.31:0/3098728378 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0048040 con 0x556a870064a0 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13a7fff640 1 -- 172.21.15.31:0/3098728378 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f13c0004290 con 0x556a870064a0 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13d145e740 1 -- 172.21.15.31:0/3098728378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c0003970 con 0x556a870064a0 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13a7fff640 1 -- 172.21.15.31:0/3098728378 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f13c0004490 con 0x556a870064a0 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13d145e740 10 client.25562 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13d145e740 1 -- 172.21.15.31:0/3098728378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556a8701fc00 con 0x556a870064a0 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.629+0000 7f13a7fff640 1 -- 172.21.15.31:0/3098728378 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c0003970 con 0x556a870064a0 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13a7fff640 1 -- 172.21.15.31:0/3098728378 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f13c004e040 con 0x556a870064a0 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13d145e740 10 client.25562 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13d145e740 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13d145e740 20 client.25562 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13d145e740 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556a87020f50 0x556a87023340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13d145e740 4 client.25562 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13d145e740 1 -- 172.21.15.31:0/3098728378 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a86f79850 con 0x556a87020f50 2024-09-16T00:45:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13c7fff640 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556a87020f50 0x556a87023340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13c7fff640 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556a87020f50 0x556a87023340 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b07d9e8d4c795bb server_cookie=4500fce0f3b1896b in_seq=0 out_seq=0 2024-09-16T00:45:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.630+0000 7f13a7fff640 10 client.25562 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13a7fff640 1 -- 172.21.15.31:0/3098728378 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a86f79850 con 0x556a87020f50 2024-09-16T00:45:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13a7fff640 10 client.25562 handle_command_reply: tid=0 2024-09-16T00:45:32.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 client.25562 shutdown 2024-09-16T00:45:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 -- 172.21.15.31:0/3098728378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 msgr2=0x556a87010fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87010fc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f13c002ae70 tx=0x7f13c002a950 comp rx=0 tx=0).stop 2024-09-16T00:45:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 -- 172.21.15.31:0/3098728378 shutdown_connections 2024-09-16T00:45:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87011680 0x556a8700e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556a87020f50 0x556a87023340 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87006dc0 0x556a8700e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 --2- 172.21.15.31:0/3098728378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87010fc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13a7fff640 0 client.25562 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 -- 172.21.15.31:0/3098728378 >> 172.21.15.31:0/3098728378 conn(0x556a87002a30 msgr2=0x556a87008710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 -- 172.21.15.31:0/3098728378 shutdown_connections 2024-09-16T00:45:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 1 -- 172.21.15.31:0/3098728378 wait complete. 2024-09-16T00:45:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.634+0000 7f13d145e740 20 client.25562 trim_cache size 0 max 16384 2024-09-16T00:45:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.651+0000 7f13d145e740 1 Processor -- start 2024-09-16T00:45:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13d145e740 1 -- start start 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13d145e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87002a30 0x556a8700d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13d145e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a870064a0 0x556a87026360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13d145e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a870268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13d145e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81aee50 con 0x556a87006dc0 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13d145e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81b16d0 con 0x556a87002a30 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13d145e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81af4c0 con 0x556a870064a0 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a870268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a870268a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33486/0 (socket says 172.21.15.31:33486) 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 -- 172.21.15.31:0/256630491 learned_addr learned my addr 172.21.15.31:0/256630491 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13c7fff640 1 --2- 172.21.15.31:0/256630491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a870064a0 0x556a87026360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 --2- 172.21.15.31:0/256630491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87002a30 0x556a8700d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 -- 172.21.15.31:0/256630491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a870064a0 msgr2=0x556a87026360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 --2- 172.21.15.31:0/256630491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a870064a0 0x556a87026360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 -- 172.21.15.31:0/256630491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87002a30 msgr2=0x556a8700d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 --2- 172.21.15.31:0/256630491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87002a30 0x556a8700d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 -- 172.21.15.31:0/256630491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a87003ae0 con 0x556a87006dc0 2024-09-16T00:45:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.652+0000 7f13cd622640 1 --2- 172.21.15.31:0/256630491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a870268a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f13c0003bc0 tx=0x7f13c002fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36be7357c8141ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13b27fc640 1 -- 172.21.15.31:0/256630491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0048040 con 0x556a87006dc0 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13b27fc640 1 -- 172.21.15.31:0/256630491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c0030e00 con 0x556a87006dc0 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13b27fc640 1 -- 172.21.15.31:0/256630491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0004630 con 0x556a87006dc0 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13d145e740 1 -- 172.21.15.31:0/256630491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 msgr2=0x556a870268a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13d145e740 1 --2- 172.21.15.31:0/256630491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a870268a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f13c0003bc0 tx=0x7f13c002fcf0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13d145e740 1 -- 172.21.15.31:0/256630491 shutdown_connections 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13d145e740 1 --2- 172.21.15.31:0/256630491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a870268a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13d145e740 1 --2- 172.21.15.31:0/256630491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a870064a0 0x556a87026360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13d145e740 1 --2- 172.21.15.31:0/256630491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a87002a30 0x556a8700d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13d145e740 1 -- 172.21.15.31:0/256630491 >> 172.21.15.31:0/256630491 conn(0x556a8700e910 msgr2=0x556a8700ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.653+0000 7f13d145e740 1 -- 172.21.15.31:0/256630491 shutdown_connections 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 1 -- 172.21.15.31:0/256630491 wait complete. 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 1 -- start start 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 1 Processor -- start 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87002a30 0x556a87026770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87023b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a87024050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81afb90 con 0x556a87006dc0 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81b1000 con 0x556a870064a0 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81b0260 con 0x556a87002a30 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13d145e740 10 client.0 _ll_register_callbacks cb 0x7f13c0046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13cd622640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a87024050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.654+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87023b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13cd622640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87002a30 0x556a87026770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87023b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:52930/0 (socket says 172.21.15.31:52930) 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13c7fff640 1 -- 172.21.15.31:0/2178096862 learned_addr learned my addr 172.21.15.31:0/2178096862 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13c7fff640 1 -- 172.21.15.31:0/2178096862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87002a30 msgr2=0x556a87026770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13c7fff640 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87002a30 0x556a87026770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13c7fff640 1 -- 172.21.15.31:0/2178096862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 msgr2=0x556a87024050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13c7fff640 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a87024050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13c7fff640 1 -- 172.21.15.31:0/2178096862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a87024590 con 0x556a870064a0 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13c7fff640 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87023b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x556a8700db00 tx=0x7f13b4004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84e73ec23ff7bdf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13a7fff640 1 -- 172.21.15.31:0/2178096862 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4027650 con 0x556a870064a0 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13d145e740 1 -- 172.21.15.31:0/2178096862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c0030e00 con 0x556a870064a0 2024-09-16T00:45:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13a7fff640 1 -- 172.21.15.31:0/2178096862 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f13b40277f0 con 0x556a870064a0 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13a7fff640 1 -- 172.21.15.31:0/2178096862 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f13b4030050 con 0x556a870064a0 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13d145e740 10 client.25568 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.655+0000 7f13d145e740 1 -- 172.21.15.31:0/2178096862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556a8708b5e0 con 0x556a870064a0 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13a7fff640 1 -- 172.21.15.31:0/2178096862 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c0030e00 con 0x556a870064a0 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13a7fff640 1 -- 172.21.15.31:0/2178096862 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f13b401e530 con 0x556a870064a0 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13d145e740 10 client.25568 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13d145e740 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13d145e740 20 client.25568 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13d145e740 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556a87011680 0x556a8708dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13d145e740 4 client.25568 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13d145e740 1 -- 172.21.15.31:0/2178096862 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a86f73d60 con 0x556a87011680 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13cd622640 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556a87011680 0x556a8708dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13a7fff640 10 client.25568 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.656+0000 7f13cd622640 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556a87011680 0x556a8708dd80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd35e6f886c960ba server_cookie=4227a2ab67b3d0e2 in_seq=0 out_seq=0 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13a7fff640 1 -- 172.21.15.31:0/2178096862 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x556a86f73d60 con 0x556a87011680 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13a7fff640 10 client.25568 handle_command_reply: tid=0 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 client.25568 shutdown 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 -- 172.21.15.31:0/2178096862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 msgr2=0x556a87023b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87023b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x556a8700db00 tx=0x7f13b4004560 comp rx=0 tx=0).stop 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 -- 172.21.15.31:0/2178096862 shutdown_connections 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a87006dc0 0x556a87024050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a870064a0 0x556a87023b10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x556a87011680 0x556a8708dd80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 --2- 172.21.15.31:0/2178096862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a87002a30 0x556a87026770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13a7fff640 0 client.25568 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 -- 172.21.15.31:0/2178096862 >> 172.21.15.31:0/2178096862 conn(0x556a8700e910 msgr2=0x556a8700d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 -- 172.21.15.31:0/2178096862 shutdown_connections 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 1 -- 172.21.15.31:0/2178096862 wait complete. 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.659+0000 7f13d145e740 20 client.25568 trim_cache size 0 max 16384 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (8682 inodes in the stack)", 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "4be73a9d-ac52-4ff9-86f0-07ecc97fcfc9": { 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "4be73a9d-ac52-4ff9-86f0-07ecc97fcfc9", 2024-09-16T00:45:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:45:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:32.664 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:45:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 -- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13b80a3240 msgr2=0x7f13b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13b80a3240 0x7f13b80a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0cdb10 tx=0x7f13bc0d3b30 comp rx=0 tx=0).stop 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 -- 172.21.15.31:0/2052556137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13c8102d40 msgr2=0x7f13c819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13c8102d40 0x7f13c819a5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00e9f0 tx=0x7f13bc00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 -- 172.21.15.31:0/2052556137 shutdown_connections 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13c810c170 0x7f13c819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c8103660 0x7f13c819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13b80a3240 0x7f13b80a5700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 --2- 172.21.15.31:0/2052556137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13c8102d40 0x7f13c819a5d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 -- 172.21.15.31:0/2052556137 >> 172.21.15.31:0/2052556137 conn(0x7f13c80fc7a0 msgr2=0x7f13c81008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 -- 172.21.15.31:0/2052556137 shutdown_connections 2024-09-16T00:45:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:32.660+0000 7f13cf085640 1 -- 172.21.15.31:0/2052556137 wait complete. 2024-09-16T00:45:33.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:32 smithi031 ceph-mon[30956]: pgmap v488: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.5 MiB/s wr, 964 op/s 2024-09-16T00:45:33.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:32 smithi081 ceph-mon[35539]: pgmap v488: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.5 MiB/s wr, 964 op/s 2024-09-16T00:45:33.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:32 smithi102 ceph-mon[35364]: pgmap v488: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.5 MiB/s wr, 964 op/s 2024-09-16T00:45:33.453 INFO:tasks.cephfs.filesystem:scrub status for tag:4be73a9d-ac52-4ff9-86f0-07ecc97fcfc9 - {'path': '/', 'tag': '4be73a9d-ac52-4ff9-86f0-07ecc97fcfc9', 'options': 'recursive,force'} 2024-09-16T00:45:35.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:34 smithi031 ceph-mon[30956]: pgmap v489: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 1.05k op/s 2024-09-16T00:45:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:34 smithi081 ceph-mon[35539]: pgmap v489: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 1.05k op/s 2024-09-16T00:45:35.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:34 smithi102 ceph-mon[35364]: pgmap v489: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 1.05k op/s 2024-09-16T00:45:36.188 INFO:tasks.workunit.client.0.smithi031.stdout: 95 0 0 0 0 31 0 2024-09-16T00:45:37.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:36 smithi031 ceph-mon[30956]: pgmap v490: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 781 op/s 2024-09-16T00:45:37.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:36 smithi081 ceph-mon[35539]: pgmap v490: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 781 op/s 2024-09-16T00:45:37.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:36 smithi102 ceph-mon[35364]: pgmap v490: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 781 op/s 2024-09-16T00:45:39.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:38 smithi031 ceph-mon[30956]: pgmap v491: 129 pgs: 129 active+clean; 549 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 744 op/s 2024-09-16T00:45:39.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:38 smithi081 ceph-mon[35539]: pgmap v491: 129 pgs: 129 active+clean; 549 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 744 op/s 2024-09-16T00:45:39.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:38 smithi102 ceph-mon[35364]: pgmap v491: 129 pgs: 129 active+clean; 549 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 744 op/s 2024-09-16T00:45:41.101 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:40 smithi102 ceph-mon[35364]: pgmap v492: 129 pgs: 129 active+clean; 554 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 583 op/s 2024-09-16T00:45:41.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:40 smithi031 ceph-mon[30956]: pgmap v492: 129 pgs: 129 active+clean; 554 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 583 op/s 2024-09-16T00:45:41.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:40 smithi081 ceph-mon[35539]: pgmap v492: 129 pgs: 129 active+clean; 554 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 583 op/s 2024-09-16T00:45:43.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:42 smithi031 ceph-mon[30956]: pgmap v493: 129 pgs: 129 active+clean; 554 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 815 KiB/s wr, 549 op/s 2024-09-16T00:45:43.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:42 smithi081 ceph-mon[35539]: pgmap v493: 129 pgs: 129 active+clean; 554 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 815 KiB/s wr, 549 op/s 2024-09-16T00:45:43.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:42 smithi102 ceph-mon[35364]: pgmap v493: 129 pgs: 129 active+clean; 554 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 815 KiB/s wr, 549 op/s 2024-09-16T00:45:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd168ff9640 1 -- 172.21.15.31:0/269415176 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564242fb0d20 con 0x5642430d2830 2024-09-16T00:45:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd168ff9640 10 client.25514 handle_command_reply: tid=0 2024-09-16T00:45:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 client.25514 shutdown 2024-09-16T00:45:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 -- 172.21.15.31:0/269415176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 msgr2=0x564243056f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x564243056f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0cd7e0 tx=0x7fd17c0a4f20 comp rx=0 tx=0).stop 2024-09-16T00:45:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 -- 172.21.15.31:0/269415176 shutdown_connections 2024-09-16T00:45:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5642430d2830 0x5642430d4c50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56424304e940 0x5642430489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564243044080 0x564243048480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd168ff9640 0 client.25514 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:43.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 --2- 172.21.15.31:0/269415176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564243043760 0x564243056f00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 -- 172.21.15.31:0/269415176 >> 172.21.15.31:0/269415176 conn(0x56424303fcf0 msgr2=0x56424304c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:43.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 -- 172.21.15.31:0/269415176 shutdown_connections 2024-09-16T00:45:43.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 1 -- 172.21.15.31:0/269415176 wait complete. 2024-09-16T00:45:43.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.804+0000 7fd189a8b740 20 client.25514 trim_cache size 0 max 16384 2024-09-16T00:45:43.806 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:45:43.806 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:45:43.806 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 14, 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "15f06940": { 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 14, 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 133.8, 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:45:43.807 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:43.808 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 -- 172.21.15.31:0/1399697645 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd16c0a32c0 msgr2=0x7fd16c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd16c0a32c0 0x7fd16c0a5780 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd178009fd0 tx=0x7fd178009500 comp rx=0 tx=0).stop 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 -- 172.21.15.31:0/1399697645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074510 msgr2=0x7fd18019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074510 0x7fd18019a6f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e9f0 tx=0x7fd17c00edf0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 -- 172.21.15.31:0/1399697645 shutdown_connections 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18010e450 0x7fd18019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd16c0a32c0 0x7fd16c0a5780 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd180074e30 0x7fd18019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 --2- 172.21.15.31:0/1399697645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd180074510 0x7fd18019a6f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 -- 172.21.15.31:0/1399697645 >> 172.21.15.31:0/1399697645 conn(0x7fd1800fd8f0 msgr2=0x7fd1800fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:43.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 -- 172.21.15.31:0/1399697645 shutdown_connections 2024-09-16T00:45:43.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:43.805+0000 7fd1876b2640 1 -- 172.21.15.31:0/1399697645 wait complete. 2024-09-16T00:45:44.616 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '15f06940', quiesce took 29.8 seconds. Will release after: -16.0 2024-09-16T00:45:44.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:45:44.886 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:45.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:44 smithi031 ceph-mon[30956]: pgmap v494: 129 pgs: 129 active+clean; 562 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 606 op/s 2024-09-16T00:45:45.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:44 smithi081 ceph-mon[35539]: pgmap v494: 129 pgs: 129 active+clean; 562 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 606 op/s 2024-09-16T00:45:45.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:44 smithi102 ceph-mon[35364]: pgmap v494: 129 pgs: 129 active+clean; 562 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 606 op/s 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 -- 172.21.15.31:0/332535308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8100e20 msgr2=0x7fbda8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/332535308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8100e20 0x7fbda8101200 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009900 tx=0x7fbd9802a480 comp rx=0 tx=0).stop 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 -- 172.21.15.31:0/332535308 shutdown_connections 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/332535308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbda8102ef0 0x7fbda810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/332535308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbda8102570 0x7fbda81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/332535308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8100e20 0x7fbda8101200 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 -- 172.21.15.31:0/332535308 >> 172.21.15.31:0/332535308 conn(0x7fbda80fc7d0 msgr2=0x7fbda80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 -- 172.21.15.31:0/332535308 shutdown_connections 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 -- 172.21.15.31:0/332535308 wait complete. 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.404+0000 7fbdaca5f640 1 Processor -- start 2024-09-16T00:45:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbdaca5f640 1 -- start start 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbdaca5f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbda8100e20 0x7fbda81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbdaca5f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbda8102570 0x7fbda8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbdaca5f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8102ef0 0x7fbda819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbdaca5f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8110f80 con 0x7fbda8102ef0 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbdaca5f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8110e00 con 0x7fbda8102570 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbdaca5f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8111100 con 0x7fbda8100e20 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda5d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbda8102570 0x7fbda8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8102ef0 0x7fbda819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda5d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbda8102570 0x7fbda8198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:57534/0 (socket says 172.21.15.31:57534) 2024-09-16T00:45:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8102ef0 0x7fbda819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60092/0 (socket says 172.21.15.31:60092) 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda5d9b640 1 -- 172.21.15.31:0/3789316501 learned_addr learned my addr 172.21.15.31:0/3789316501 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda659c640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbda8100e20 0x7fbda81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda5d9b640 1 -- 172.21.15.31:0/3789316501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbda8100e20 msgr2=0x7fbda81985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda5d9b640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbda8100e20 0x7fbda81985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda5d9b640 1 -- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8102ef0 msgr2=0x7fbda819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda5d9b640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8102ef0 0x7fbda819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda5d9b640 1 -- 172.21.15.31:0/3789316501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda819d550 con 0x7fbda8102570 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda6d9d640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8102ef0 0x7fbda819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.405+0000 7fbda5d9b640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbda8102570 0x7fbda8198ae0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ea60 tx=0x7fbd9c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.406+0000 7fbd977fe640 1 -- 172.21.15.31:0/3789316501 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00cde0 con 0x7fbda8102570 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.406+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda819d840 con 0x7fbda8102570 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.406+0000 7fbd977fe640 1 -- 172.21.15.31:0/3789316501 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9c0035f0 con 0x7fbda8102570 2024-09-16T00:45:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.406+0000 7fbd977fe640 1 -- 172.21.15.31:0/3789316501 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c010670 con 0x7fbda8102570 2024-09-16T00:45:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.406+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda81a5120 con 0x7fbda8102570 2024-09-16T00:45:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.407+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda8101200 con 0x7fbda8102570 2024-09-16T00:45:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.408+0000 7fbd977fe640 1 -- 172.21.15.31:0/3789316501 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbd9c0037a0 con 0x7fbda8102570 2024-09-16T00:45:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.409+0000 7fbd977fe640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbd900a3240 0x7fbd900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.409+0000 7fbda659c640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbd900a3240 0x7fbd900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.409+0000 7fbd977fe640 1 -- 172.21.15.31:0/3789316501 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fbd9c0d0290 con 0x7fbda8102570 2024-09-16T00:45:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.409+0000 7fbda659c640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbd900a3240 0x7fbd900a5700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009230 tx=0x7fbd980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.412+0000 7fbd977fe640 1 -- 172.21.15.31:0/3789316501 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd9c09cc00 con 0x7fbda8102570 2024-09-16T00:45:45.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.650+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbda8199900 con 0x7fbda8102570 2024-09-16T00:45:45.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.651+0000 7fbd977fe640 1 -- 172.21.15.31:0/3789316501 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fbda8199900 con 0x7fbda8102570 2024-09-16T00:45:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:45:45.652 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:45:45.652 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:45:45.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbd900a3240 msgr2=0x7fbd900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbd900a3240 0x7fbd900a5700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009230 tx=0x7fbd980058d0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbda8102570 msgr2=0x7fbda8198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:45.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbda8102570 0x7fbda8198ae0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ea60 tx=0x7fbd9c00ef20 comp rx=0 tx=0).stop 2024-09-16T00:45:45.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 shutdown_connections 2024-09-16T00:45:45.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda8102ef0 0x7fbda819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbda8102570 0x7fbda8198ae0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbd900a3240 0x7fbd900a5700 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 --2- 172.21.15.31:0/3789316501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbda8100e20 0x7fbda81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:45.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.653+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 >> 172.21.15.31:0/3789316501 conn(0x7fbda80fc7d0 msgr2=0x7fbda8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:45.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.654+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 shutdown_connections 2024-09-16T00:45:45.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:45.654+0000 7fbdaca5f640 1 -- 172.21.15.31:0/3789316501 wait complete. 2024-09-16T00:45:46.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:45 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3789316501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:46.188 INFO:tasks.workunit.client.0.smithi031.stdout: 95 1869 0 324 0 1021 0 2024-09-16T00:45:46.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:45 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3789316501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:46.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:45 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3789316501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:45:46.471 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 15f06940 --release --await' 2024-09-16T00:45:46.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 15f06940 --release --await 2024-09-16T00:45:46.752 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:45:47.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:46 smithi031 ceph-mon[30956]: pgmap v495: 129 pgs: 129 active+clean; 554 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 465 op/s 2024-09-16T00:45:47.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:46 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:45:47.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:46 smithi081 ceph-mon[35539]: pgmap v495: 129 pgs: 129 active+clean; 554 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 465 op/s 2024-09-16T00:45:47.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:46 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:45:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.266+0000 7f441caa5640 1 -- 172.21.15.31:0/2311686188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4418103740 msgr2=0x7f4418107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.266+0000 7f441caa5640 1 --2- 172.21.15.31:0/2311686188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4418103740 0x7f4418107b80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f440c009890 tx=0x7f440c02a380 comp rx=0 tx=0).stop 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.266+0000 7f441caa5640 1 -- 172.21.15.31:0/2311686188 shutdown_connections 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.266+0000 7f441caa5640 1 --2- 172.21.15.31:0/2311686188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4418108290 0x7f441810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.266+0000 7f441caa5640 1 --2- 172.21.15.31:0/2311686188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4418103740 0x7f4418107b80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.266+0000 7f441caa5640 1 --2- 172.21.15.31:0/2311686188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4418102d90 0x7f4418103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.266+0000 7f441caa5640 1 -- 172.21.15.31:0/2311686188 >> 172.21.15.31:0/2311686188 conn(0x7f44180fc7d0 msgr2=0x7f44180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.266+0000 7f441caa5640 1 -- 172.21.15.31:0/2311686188 shutdown_connections 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.266+0000 7f441caa5640 1 -- 172.21.15.31:0/2311686188 wait complete. 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.267+0000 7f441caa5640 1 Processor -- start 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.267+0000 7f441caa5640 1 -- start start 2024-09-16T00:45:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441caa5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4418102d90 0x7f4418198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441caa5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4418103740 0x7f4418198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441caa5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4418108290 0x7f441819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441caa5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4418112050 con 0x7f4418103740 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441caa5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4418111ed0 con 0x7f4418102d90 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441caa5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f44181121d0 con 0x7f4418108290 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f4415d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4418103740 0x7f4418198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f4415d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4418103740 0x7f4418198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60104/0 (socket says 172.21.15.31:60104) 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4418102d90 0x7f4418198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f4415d9b640 1 -- 172.21.15.31:0/3405672018 learned_addr learned my addr 172.21.15.31:0/3405672018 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f4416d9d640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4418108290 0x7f441819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441659c640 1 -- 172.21.15.31:0/3405672018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4418108290 msgr2=0x7f441819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441659c640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4418108290 0x7f441819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441659c640 1 -- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4418103740 msgr2=0x7f4418198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441659c640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4418103740 0x7f4418198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f441659c640 1 -- 172.21.15.31:0/3405672018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441819d5e0 con 0x7f4418102d90 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.268+0000 7f4415d9b640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4418103740 0x7f4418198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.269+0000 7f441659c640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4418102d90 0x7f4418198630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4408005fd0 tx=0x7f440800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.269+0000 7f44077fe640 1 -- 172.21.15.31:0/3405672018 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440800cd00 con 0x7f4418102d90 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.269+0000 7f44077fe640 1 -- 172.21.15.31:0/3405672018 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44080035f0 con 0x7f4418102d90 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.269+0000 7f441caa5640 1 -- 172.21.15.31:0/3405672018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441819d8d0 con 0x7f4418102d90 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.269+0000 7f44077fe640 1 -- 172.21.15.31:0/3405672018 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408010690 con 0x7f4418102d90 2024-09-16T00:45:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.269+0000 7f441caa5640 1 -- 172.21.15.31:0/3405672018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44181a51b0 con 0x7f4418102d90 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.271+0000 7f44077fe640 1 -- 172.21.15.31:0/3405672018 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f44080037a0 con 0x7f4418102d90 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.271+0000 7f441ee7e740 1 Processor -- start 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f441ee7e740 1 -- start start 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183ac1060 0x560183ac1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183acc240 0x560183ace630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f441ee7e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560183a26620 con 0x560183ac1060 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f441ee7e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5601839cde50 con 0x560183ac1980 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f441ee7e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560183a225e0 con 0x560183acc240 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f4416d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183ac1060 0x560183ac1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f441659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f4415d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183acc240 0x560183ace630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f4415d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183acc240 0x560183ace630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:49240/0 (socket says 172.21.15.31:49240) 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.272+0000 7f4415d9b640 1 -- 172.21.15.31:0/815722422 learned_addr learned my addr 172.21.15.31:0/815722422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4415d9b640 1 -- 172.21.15.31:0/815722422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 msgr2=0x560183acbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4415d9b640 1 --2- 172.21.15.31:0/815722422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4415d9b640 1 -- 172.21.15.31:0/815722422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183ac1060 msgr2=0x560183ac1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4415d9b640 1 --2- 172.21.15.31:0/815722422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183ac1060 0x560183ac1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4415d9b640 1 -- 172.21.15.31:0/815722422 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601839bed40 con 0x560183acc240 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f44077fe640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44000a3240 0x7f44000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4415d9b640 1 --2- 172.21.15.31:0/815722422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183acc240 0x560183ace630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f440c02a970 tx=0x7f440c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8abe2448e78a31f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f44077fe640 1 -- 172.21.15.31:0/3405672018 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f44080cff90 con 0x7f4418102d90 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4416d9d640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44000a3240 0x7f44000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4404ff9640 1 -- 172.21.15.31:0/815722422 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c037040 con 0x560183acc240 2024-09-16T00:45:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4404ff9640 1 -- 172.21.15.31:0/815722422 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440c004230 con 0x560183acc240 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.273+0000 7f4404ff9640 1 -- 172.21.15.31:0/815722422 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c003cd0 con 0x560183acc240 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.274+0000 7f441ee7e740 1 -- 172.21.15.31:0/815722422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183acc240 msgr2=0x560183ace630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.274+0000 7f441ee7e740 1 --2- 172.21.15.31:0/815722422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183acc240 0x560183ace630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f440c02a970 tx=0x7f440c009510 comp rx=0 tx=0).stop 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.274+0000 7f4416d9d640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44000a3240 0x7f44000a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4410005fd0 tx=0x7f4410009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.274+0000 7f441ee7e740 1 -- 172.21.15.31:0/815722422 shutdown_connections 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.274+0000 7f441ee7e740 1 --2- 172.21.15.31:0/815722422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183acc240 0x560183ace630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.274+0000 7f441ee7e740 1 --2- 172.21.15.31:0/815722422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.274+0000 7f441ee7e740 1 --2- 172.21.15.31:0/815722422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183ac1060 0x560183ac1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.274+0000 7f441ee7e740 1 -- 172.21.15.31:0/815722422 >> 172.21.15.31:0/815722422 conn(0x560183abd5f0 msgr2=0x560183abd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.274+0000 7f441ee7e740 1 -- 172.21.15.31:0/815722422 shutdown_connections 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.275+0000 7f441ee7e740 1 -- 172.21.15.31:0/815722422 wait complete. 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.275+0000 7f441ee7e740 1 -- start start 2024-09-16T00:45:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.275+0000 7f441ee7e740 1 Processor -- start 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.275+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.275+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ac9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.275+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ac9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.275+0000 7f441ee7e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560183acee10 con 0x560183acc240 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.275+0000 7f441ee7e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5601839aef10 con 0x560183ac1980 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.275+0000 7f441ee7e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560183acef90 con 0x560183ac1060 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441ee7e740 10 client.0 _ll_register_callbacks cb 0x5601839d2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:49242/0 (socket says 172.21.15.31:49242) 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441659c640 1 -- 172.21.15.31:0/3739665701 learned_addr learned my addr 172.21.15.31:0/3739665701 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f4415d9b640 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ac9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f4416d9d640 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ac9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441659c640 1 -- 172.21.15.31:0/3739665701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 msgr2=0x560183ac9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441659c640 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ac9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441659c640 1 -- 172.21.15.31:0/3739665701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 msgr2=0x560183ac9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441659c640 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ac9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441659c640 1 -- 172.21.15.31:0/3739665701 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560183735580 con 0x560183ac1060 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441659c640 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac9130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4408009230 tx=0x7f440809c030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15336deef21f6b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f43fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f43fdffb640 1 -- 172.21.15.31:0/3739665701 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440809c8a0 con 0x560183ac1060 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f441ee7e740 1 -- 172.21.15.31:0/3739665701 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440c004230 con 0x560183ac1060 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f43fdffb640 1 -- 172.21.15.31:0/3739665701 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f44080bc0d0 con 0x560183ac1060 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.276+0000 7f43fdffb640 1 -- 172.21.15.31:0/3739665701 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f440809c1a0 con 0x560183ac1060 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f441ee7e740 10 client.25525 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f441ee7e740 1 -- 172.21.15.31:0/3739665701 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5601837ff960 con 0x560183ac1060 2024-09-16T00:45:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f43fdffb640 1 -- 172.21.15.31:0/3739665701 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440c004230 con 0x560183ac1060 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f43fdffb640 1 -- 172.21.15.31:0/3739665701 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f44080a4980 con 0x560183ac1060 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f441ee7e740 10 client.25525 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f441ee7e740 10 client.25525 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f441ee7e740 20 client.25525 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f441ee7e740 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x560183adbd50 0x560183ade160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f441ee7e740 4 client.25525 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f441ee7e740 1 -- 172.21.15.31:0/3739665701 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560183a33b90 con 0x560183adbd50 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.277+0000 7f4415d9b640 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x560183adbd50 0x560183ade160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.278+0000 7f43fdffb640 10 client.25525 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.278+0000 7f4415d9b640 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x560183adbd50 0x560183ade160 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a186bc08a6e406 server_cookie=b431298c3e7bae23 in_seq=0 out_seq=0 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f43fdffb640 1 -- 172.21.15.31:0/3739665701 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560183a33b90 con 0x560183adbd50 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f43fdffb640 10 client.25525 handle_command_reply: tid=0 2024-09-16T00:45:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 client.25525 shutdown 2024-09-16T00:45:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 -- 172.21.15.31:0/3739665701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 msgr2=0x560183ac9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac9130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4408009230 tx=0x7f440809c030 comp rx=0 tx=0).stop 2024-09-16T00:45:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 -- 172.21.15.31:0/3739665701 shutdown_connections 2024-09-16T00:45:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ac9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ac9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x560183adbd50 0x560183ade160 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 --2- 172.21.15.31:0/3739665701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac9130 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f43fdffb640 0 client.25525 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 -- 172.21.15.31:0/3739665701 >> 172.21.15.31:0/3739665701 conn(0x560183abd5f0 msgr2=0x560183ac3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 -- 172.21.15.31:0/3739665701 shutdown_connections 2024-09-16T00:45:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.279+0000 7f441ee7e740 1 -- 172.21.15.31:0/3739665701 wait complete. 2024-09-16T00:45:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.280+0000 7f441ee7e740 20 client.25525 trim_cache size 0 max 16384 2024-09-16T00:45:47.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:46 smithi102 ceph-mon[35364]: pgmap v495: 129 pgs: 129 active+clean; 554 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 465 op/s 2024-09-16T00:45:47.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:46 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:45:47.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.299+0000 7f441ee7e740 1 Processor -- start 2024-09-16T00:45:47.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.299+0000 7f441ee7e740 1 -- start start 2024-09-16T00:45:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.299+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.299+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.299+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ae3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.299+0000 7f441ee7e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f44181aab90 con 0x560183acc240 2024-09-16T00:45:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.299+0000 7f441ee7e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f44181ad410 con 0x560183ac1980 2024-09-16T00:45:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.299+0000 7f441ee7e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f44181ab200 con 0x560183ac1060 2024-09-16T00:45:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4415d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4415d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:45946/0 (socket says 172.21.15.31:45946) 2024-09-16T00:45:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4415d9b640 1 -- 172.21.15.31:0/1227802218 learned_addr learned my addr 172.21.15.31:0/1227802218 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f441659c640 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4416d9d640 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ae3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4415d9b640 1 -- 172.21.15.31:0/1227802218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 msgr2=0x560183ac3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4415d9b640 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4415d9b640 1 -- 172.21.15.31:0/1227802218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 msgr2=0x560183ae3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4415d9b640 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ae3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4415d9b640 1 -- 172.21.15.31:0/1227802218 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560183ac4d30 con 0x560183ac1980 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4416d9d640 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ae3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.300+0000 7f4415d9b640 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acb0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f440c005d90 tx=0x7f440c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bfb6cecdf6c7a84 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.301+0000 7f43fffff640 1 -- 172.21.15.31:0/1227802218 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c037040 con 0x560183ac1980 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.301+0000 7f43fffff640 1 -- 172.21.15.31:0/1227802218 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440c053070 con 0x560183ac1980 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.301+0000 7f43fffff640 1 -- 172.21.15.31:0/1227802218 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c004170 con 0x560183ac1980 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.301+0000 7f441ee7e740 1 -- 172.21.15.31:0/1227802218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 msgr2=0x560183acb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.301+0000 7f441ee7e740 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acb0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f440c005d90 tx=0x7f440c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.302+0000 7f441ee7e740 1 -- 172.21.15.31:0/1227802218 shutdown_connections 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.302+0000 7f441ee7e740 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ae3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.302+0000 7f441ee7e740 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183acb0a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.302+0000 7f441ee7e740 1 --2- 172.21.15.31:0/1227802218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ac3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.302+0000 7f441ee7e740 1 -- 172.21.15.31:0/1227802218 >> 172.21.15.31:0/1227802218 conn(0x560183abd5f0 msgr2=0x560183aca990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.302+0000 7f441ee7e740 1 -- 172.21.15.31:0/1227802218 shutdown_connections 2024-09-16T00:45:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.302+0000 7f441ee7e740 1 -- 172.21.15.31:0/1227802218 wait complete. 2024-09-16T00:45:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.302+0000 7f441ee7e740 1 -- start start 2024-09-16T00:45:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.302+0000 7f441ee7e740 1 Processor -- start 2024-09-16T00:45:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ad41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ad4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441ee7e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ad11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441ee7e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f44181ab8d0 con 0x560183acc240 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441ee7e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f44181acd40 con 0x560183ac1980 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441ee7e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f44181abfa0 con 0x560183ac1060 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441ee7e740 10 client.0 _ll_register_callbacks cb 0x7f4408098440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4415d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ad4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4416d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ad11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ad41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ad41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:59876/0 (socket says 172.21.15.31:59876) 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f441659c640 1 -- 172.21.15.31:0/2380609677 learned_addr learned my addr 172.21.15.31:0/2380609677 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4415d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ad4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:45950/0 (socket says 172.21.15.31:45950) 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4416d9d640 1 -- 172.21.15.31:0/2380609677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 msgr2=0x560183ad41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4416d9d640 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ad41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4416d9d640 1 -- 172.21.15.31:0/2380609677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 msgr2=0x560183ad4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4416d9d640 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ad4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4416d9d640 1 -- 172.21.15.31:0/2380609677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560183ad1720 con 0x560183acc240 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4415d9b640 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ad4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f4416d9d640 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ad11b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4410011db0 tx=0x7f4410004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a649944bb8410ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.303+0000 7f43fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f43fdffb640 1 -- 172.21.15.31:0/2380609677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410004b20 con 0x560183acc240 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f43fdffb640 1 -- 172.21.15.31:0/2380609677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f4410004cc0 con 0x560183acc240 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f441ee7e740 1 -- 172.21.15.31:0/2380609677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440c053070 con 0x560183acc240 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f43fdffb640 1 -- 172.21.15.31:0/2380609677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4410034050 con 0x560183acc240 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f441ee7e740 10 client.15921 fetch_fsmap learned FSMap version 20 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f441ee7e740 1 -- 172.21.15.31:0/2380609677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560183b472c0 con 0x560183acc240 2024-09-16T00:45:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f43fdffb640 1 -- 172.21.15.31:0/2380609677 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440c053070 con 0x560183acc240 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f43fdffb640 1 -- 172.21.15.31:0/2380609677 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f44100056d0 con 0x560183acc240 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f441ee7e740 10 client.15921 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f441ee7e740 10 client.15921 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f441ee7e740 20 client.15921 populate_metadata read hostname 'smithi031' 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f441ee7e740 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x560183b47a90 0x560183b49eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f441ee7e740 4 client.15921 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "15f06940", "release": true, "await": true}] 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.304+0000 7f441ee7e740 1 -- 172.21.15.31:0/2380609677 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "set_id": "15f06940", "release": true, "await": true}) -- 0x560183a2e0a0 con 0x560183b47a90 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.305+0000 7f441659c640 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x560183b47a90 0x560183b49eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.305+0000 7f441659c640 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x560183b47a90 0x560183b49eb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3205e8657ba1d473 server_cookie=b40c42415175714f in_seq=0 out_seq=0 2024-09-16T00:45:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:47.305+0000 7f43fdffb640 10 client.15921 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f43fdffb640 1 -- 172.21.15.31:0/2380609677 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560183a2e0a0 con 0x560183b47a90 2024-09-16T00:45:48.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f43fdffb640 10 client.15921 handle_command_reply: tid=0 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f441ee7e740 1 client.15921 shutdown 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f441ee7e740 1 -- 172.21.15.31:0/2380609677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 msgr2=0x560183ad11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f441ee7e740 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ad11b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4410011db0 tx=0x7f4410004a40 comp rx=0 tx=0).stop 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f441ee7e740 1 -- 172.21.15.31:0/2380609677 shutdown_connections 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f441ee7e740 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560183acc240 0x560183ad11b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f441ee7e740 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560183ac1980 0x560183ad4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f441ee7e740 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x560183b47a90 0x560183b49eb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f441ee7e740 1 --2- 172.21.15.31:0/2380609677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560183ac1060 0x560183ad41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f43fdffb640 0 client.15921 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.181+0000 7f441ee7e740 1 -- 172.21.15.31:0/2380609677 >> 172.21.15.31:0/2380609677 conn(0x560183abd5f0 msgr2=0x560183ac9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441ee7e740 1 -- 172.21.15.31:0/2380609677 shutdown_connections 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441ee7e740 1 -- 172.21.15.31:0/2380609677 wait complete. 2024-09-16T00:45:48.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441ee7e740 20 client.15921 trim_cache size 0 max 16384 2024-09-16T00:45:48.184 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:45:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:45:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:45:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 16, 2024-09-16T00:45:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:45:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "15f06940": { 2024-09-16T00:45:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 16, 2024-09-16T00:45:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:45:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 133.8, 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:45:48.186 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 -- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44000a3240 msgr2=0x7f44000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44000a3240 0x7f44000a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4410005fd0 tx=0x7f4410009500 comp rx=0 tx=0).stop 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 -- 172.21.15.31:0/3405672018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4418102d90 msgr2=0x7f4418198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4418102d90 0x7f4418198630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4408005fd0 tx=0x7f440800ee50 comp rx=0 tx=0).stop 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 -- 172.21.15.31:0/3405672018 shutdown_connections 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f44000a3240 0x7f44000a5700 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4418108290 0x7f441819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4418103740 0x7f4418198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 --2- 172.21.15.31:0/3405672018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4418102d90 0x7f4418198630 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.182+0000 7f441caa5640 1 -- 172.21.15.31:0/3405672018 >> 172.21.15.31:0/3405672018 conn(0x7f44180fc7d0 msgr2=0x7f4418105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.183+0000 7f441caa5640 1 -- 172.21.15.31:0/3405672018 shutdown_connections 2024-09-16T00:45:48.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:45:48.183+0000 7f441caa5640 1 -- 172.21.15.31:0/3405672018 wait complete. 2024-09-16T00:45:49.001 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '15f06940', total seconds elapsed: 34.1 2024-09-16T00:45:49.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:48 smithi031 ceph-mon[30956]: pgmap v496: 129 pgs: 129 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 584 op/s 2024-09-16T00:45:49.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:48 smithi081 ceph-mon[35539]: pgmap v496: 129 pgs: 129 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 584 op/s 2024-09-16T00:45:49.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:48 smithi102 ceph-mon[35364]: pgmap v496: 129 pgs: 129 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 584 op/s 2024-09-16T00:45:50.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:49 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:45:50.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:49 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:45:50.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:49 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:45:51.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:50 smithi031 ceph-mon[30956]: pgmap v497: 129 pgs: 129 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 519 op/s 2024-09-16T00:45:51.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:50 smithi081 ceph-mon[35539]: pgmap v497: 129 pgs: 129 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 519 op/s 2024-09-16T00:45:51.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:50 smithi102 ceph-mon[35364]: pgmap v497: 129 pgs: 129 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 519 op/s 2024-09-16T00:45:53.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:52 smithi031 ceph-mon[30956]: pgmap v498: 129 pgs: 129 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 441 op/s 2024-09-16T00:45:53.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:52 smithi081 ceph-mon[35539]: pgmap v498: 129 pgs: 129 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 441 op/s 2024-09-16T00:45:53.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:52 smithi102 ceph-mon[35364]: pgmap v498: 129 pgs: 129 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 441 op/s 2024-09-16T00:45:55.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:54 smithi031 ceph-mon[30956]: pgmap v499: 129 pgs: 129 active+clean; 547 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 504 op/s 2024-09-16T00:45:55.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:54 smithi081 ceph-mon[35539]: pgmap v499: 129 pgs: 129 active+clean; 547 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 504 op/s 2024-09-16T00:45:55.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:54 smithi102 ceph-mon[35364]: pgmap v499: 129 pgs: 129 active+clean; 547 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 504 op/s 2024-09-16T00:45:56.188 INFO:tasks.workunit.client.0.smithi031.stdout: 96 12035 232 8339 53 9755 218 2024-09-16T00:45:57.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:56 smithi031 ceph-mon[30956]: pgmap v500: 129 pgs: 129 active+clean; 548 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 607 KiB/s wr, 434 op/s 2024-09-16T00:45:57.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:56 smithi081 ceph-mon[35539]: pgmap v500: 129 pgs: 129 active+clean; 548 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 607 KiB/s wr, 434 op/s 2024-09-16T00:45:57.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:56 smithi102 ceph-mon[35364]: pgmap v500: 129 pgs: 129 active+clean; 548 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 607 KiB/s wr, 434 op/s 2024-09-16T00:45:59.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:58 smithi031 ceph-mon[30956]: pgmap v501: 129 pgs: 129 active+clean; 549 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 833 KiB/s wr, 543 op/s 2024-09-16T00:45:59.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:58 smithi081 ceph-mon[35539]: pgmap v501: 129 pgs: 129 active+clean; 549 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 833 KiB/s wr, 543 op/s 2024-09-16T00:45:59.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:58 smithi102 ceph-mon[35364]: pgmap v501: 129 pgs: 129 active+clean; 549 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 833 KiB/s wr, 543 op/s 2024-09-16T00:46:00.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:46:00.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:46:00.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:46:00.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:45:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:46:00.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:46:00.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:46:00.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:46:00.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:45:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:46:00.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:46:00.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:46:00.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:46:00.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:45:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:46:01.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:01 smithi102 ceph-mon[35364]: pgmap v502: 129 pgs: 129 active+clean; 553 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 927 KiB/s wr, 442 op/s 2024-09-16T00:46:01.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:01 smithi031 ceph-mon[30956]: pgmap v502: 129 pgs: 129 active+clean; 553 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 927 KiB/s wr, 442 op/s 2024-09-16T00:46:01.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:01 smithi081 ceph-mon[35539]: pgmap v502: 129 pgs: 129 active+clean; 553 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 927 KiB/s wr, 442 op/s 2024-09-16T00:46:03.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:46:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:03.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:03 smithi102 ceph-mon[35364]: pgmap v503: 129 pgs: 129 active+clean; 558 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.4 MiB/s wr, 507 op/s 2024-09-16T00:46:03.846 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:03 smithi031 ceph-mon[30956]: pgmap v503: 129 pgs: 129 active+clean; 558 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.4 MiB/s wr, 507 op/s 2024-09-16T00:46:03.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:03 smithi081 ceph-mon[35539]: pgmap v503: 129 pgs: 129 active+clean; 558 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.4 MiB/s wr, 507 op/s 2024-09-16T00:46:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.257+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/1865866265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fdc100a80 msgr2=0x7f4fdc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.257+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/1865866265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fdc100a80 0x7f4fdc100ee0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009930 tx=0x7f4fc402a420 comp rx=0 tx=0).stop 2024-09-16T00:46:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.257+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/1865866265 shutdown_connections 2024-09-16T00:46:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.257+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/1865866265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fdc101420 0x7f4fdc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.257+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/1865866265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fdc100a80 0x7f4fdc100ee0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.257+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/1865866265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fdc106a80 0x7f4fdc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.257+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/1865866265 >> 172.21.15.31:0/1865866265 conn(0x7f4fdc0fc7a0 msgr2=0x7f4fdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.257+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/1865866265 shutdown_connections 2024-09-16T00:46:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.257+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/1865866265 wait complete. 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fe1d1e640 1 Processor -- start 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fe1d1e640 1 -- start start 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fe1d1e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fdc100a80 0x7f4fdc0fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fe1d1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fdc101420 0x7f4fdc0fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fe1d1e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fdc106a80 0x7f4fdc0fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fe1d1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc110ed0 con 0x7f4fdc101420 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fe1d1e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc110d50 con 0x7f4fdc100a80 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fe1d1e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc111050 con 0x7f4fdc106a80 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fd3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fdc101420 0x7f4fdc0fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fdb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fdc100a80 0x7f4fdc0fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fd3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fdc101420 0x7f4fdc0fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55194/0 (socket says 172.21.15.31:55194) 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fdb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fdc100a80 0x7f4fdc0fce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:45654/0 (socket says 172.21.15.31:45654) 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.258+0000 7f4fd3fff640 1 -- 172.21.15.31:0/153227861 learned_addr learned my addr 172.21.15.31:0/153227861 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fdc106a80 0x7f4fdc0fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fdb7fe640 1 -- 172.21.15.31:0/153227861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fdc106a80 msgr2=0x7f4fdc0fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fdb7fe640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fdc106a80 0x7f4fdc0fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fdb7fe640 1 -- 172.21.15.31:0/153227861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fdc101420 msgr2=0x7f4fdc0fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fdb7fe640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fdc101420 0x7f4fdc0fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fdb7fe640 1 -- 172.21.15.31:0/153227861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc1a6930 con 0x7f4fdc100a80 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fdb7fe640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fdc100a80 0x7f4fdc0fce00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00d720 tx=0x7f4fcc00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fd97fa640 1 -- 172.21.15.31:0/153227861 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc017070 con 0x7f4fdc100a80 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fd97fa640 1 -- 172.21.15.31:0/153227861 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fcc00de30 con 0x7f4fdc100a80 2024-09-16T00:46:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fd97fa640 1 -- 172.21.15.31:0/153227861 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0047a0 con 0x7f4fdc100a80 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/153227861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fdc1a6c20 con 0x7f4fdc100a80 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.259+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/153227861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fdc1a7160 con 0x7f4fdc100a80 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.261+0000 7f4fd97fa640 1 -- 172.21.15.31:0/153227861 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4fcc0099d0 con 0x7f4fdc100a80 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.262+0000 7f4fe40f7740 1 Processor -- start 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.262+0000 7f4fe40f7740 1 -- start start 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a684a0 0x561bd4a68880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a68dc0 0x561bd4a73140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a73680 0x561bd4a75a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561bd49ce2e0 con 0x561bd4a68dc0 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561bd4975ce0 con 0x561bd4a684a0 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561bd49ca2a0 con 0x561bd4a73680 2024-09-16T00:46:04.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a684a0 0x561bd4a68880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a684a0 0x561bd4a68880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:45664/0 (socket says 172.21.15.31:45664) 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdbfff640 1 -- 172.21.15.31:0/3491126335 learned_addr learned my addr 172.21.15.31:0/3491126335 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdb7fe640 1 --2- 172.21.15.31:0/3491126335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a68dc0 0x561bd4a73140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/3491126335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a73680 0x561bd4a75a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdbfff640 1 -- 172.21.15.31:0/3491126335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a73680 msgr2=0x561bd4a75a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/3491126335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a73680 0x561bd4a75a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdbfff640 1 -- 172.21.15.31:0/3491126335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a68dc0 msgr2=0x561bd4a73140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/3491126335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a68dc0 0x561bd4a73140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdbfff640 1 -- 172.21.15.31:0/3491126335 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd495a440 con 0x561bd4a684a0 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fd97fa640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4fc80a32c0 0x7f4fc80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdb7fe640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4fc80a32c0 0x7f4fc80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.263+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/3491126335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a684a0 0x561bd4a68880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009fc0 tx=0x7f4fd4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f43635285509de7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.264+0000 7f4fd97fa640 1 -- 172.21.15.31:0/153227861 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f4fcc0cf870 con 0x7f4fdc100a80 2024-09-16T00:46:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.264+0000 7f4fd1ffb640 1 -- 172.21.15.31:0/3491126335 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4003b70 con 0x561bd4a684a0 2024-09-16T00:46:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.264+0000 7f4fd1ffb640 1 -- 172.21.15.31:0/3491126335 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd4003d10 con 0x561bd4a684a0 2024-09-16T00:46:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.264+0000 7f4fd1ffb640 1 -- 172.21.15.31:0/3491126335 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4011aa0 con 0x561bd4a684a0 2024-09-16T00:46:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.264+0000 7f4fe40f7740 1 -- 172.21.15.31:0/3491126335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a684a0 msgr2=0x561bd4a68880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.264+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/3491126335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a684a0 0x561bd4a68880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009fc0 tx=0x7f4fd4010860 comp rx=0 tx=0).stop 2024-09-16T00:46:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.264+0000 7f4fdb7fe640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4fc80a32c0 0x7f4fc80a5780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc098450 tx=0x7f4fcc086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.265+0000 7f4fe40f7740 1 -- 172.21.15.31:0/3491126335 shutdown_connections 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.265+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/3491126335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a73680 0x561bd4a75a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.265+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/3491126335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a68dc0 0x561bd4a73140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.265+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/3491126335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a684a0 0x561bd4a68880 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.265+0000 7f4fe40f7740 1 -- 172.21.15.31:0/3491126335 >> 172.21.15.31:0/3491126335 conn(0x561bd4a64a30 msgr2=0x561bd4a64e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.265+0000 7f4fe40f7740 1 -- 172.21.15.31:0/3491126335 shutdown_connections 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.265+0000 7f4fe40f7740 1 -- 172.21.15.31:0/3491126335 wait complete. 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.265+0000 7f4fe40f7740 1 -- start start 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fe40f7740 1 Processor -- start 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a72f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a70220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 0x561bd4a70760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561bd4a76250 con 0x561bd4a684a0 2024-09-16T00:46:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561bd4989050 con 0x561bd4a73680 2024-09-16T00:46:04.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561bd4a763d0 con 0x561bd4a68dc0 2024-09-16T00:46:04.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fe40f7740 10 client.0 _ll_register_callbacks cb 0x561bd49c9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:04.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a72f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fd3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a70220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fd3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a70220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:54894/0 (socket says 172.21.15.31:54894) 2024-09-16T00:46:04.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a72f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55214/0 (socket says 172.21.15.31:55214) 2024-09-16T00:46:04.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.266+0000 7f4fd3fff640 1 -- 172.21.15.31:0/1963711750 learned_addr learned my addr 172.21.15.31:0/1963711750 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:04.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.267+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 0x561bd4a70760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.267+0000 7f4fd3fff640 1 -- 172.21.15.31:0/1963711750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 msgr2=0x561bd4a70760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.267+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 0x561bd4a70760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.267+0000 7f4fd3fff640 1 -- 172.21.15.31:0/1963711750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 msgr2=0x561bd4a72f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.267+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a72f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.267+0000 7f4fd3fff640 1 -- 172.21.15.31:0/1963711750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd473f3c0 con 0x561bd4a68dc0 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.267+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a72f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.267+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a70220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009900 tx=0x7f4fc40044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df1760de73575124 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.267+0000 7f4fbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.268+0000 7f4fbaffd640 1 -- 172.21.15.31:0/1963711750 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4035040 con 0x561bd4a68dc0 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.268+0000 7f4fbaffd640 1 -- 172.21.15.31:0/1963711750 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f4fc4005510 con 0x561bd4a68dc0 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.268+0000 7f4fe40f7740 1 -- 172.21.15.31:0/1963711750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd4003d10 con 0x561bd4a68dc0 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.268+0000 7f4fbaffd640 1 -- 172.21.15.31:0/1963711750 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4fc40046c0 con 0x561bd4a68dc0 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.268+0000 7f4fe40f7740 10 client.25543 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:04.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.268+0000 7f4fe40f7740 1 -- 172.21.15.31:0/1963711750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561bd4a7f770 con 0x561bd4a68dc0 2024-09-16T00:46:04.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.268+0000 7f4fbaffd640 1 -- 172.21.15.31:0/1963711750 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd4003d10 con 0x561bd4a68dc0 2024-09-16T00:46:04.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.268+0000 7f4fbaffd640 1 -- 172.21.15.31:0/1963711750 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f4fc4054020 con 0x561bd4a68dc0 2024-09-16T00:46:04.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.269+0000 7f4fe40f7740 10 client.25543 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:04.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.269+0000 7f4fe40f7740 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:46:04.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.269+0000 7f4fe40f7740 20 client.25543 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:04.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.269+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561bd4a80c80 0x561bd4a830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.269+0000 7f4fe40f7740 4 client.25543 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:46:04.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.269+0000 7f4fe40f7740 1 -- 172.21.15.31:0/1963711750 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd49db850 con 0x561bd4a80c80 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.269+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561bd4a80c80 0x561bd4a830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.269+0000 7f4fbaffd640 10 client.25543 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.269+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561bd4a80c80 0x561bd4a830a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ad95c0ebcf10900 server_cookie=92a4ed424917feb0 in_seq=0 out_seq=0 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fbaffd640 1 -- 172.21.15.31:0/1963711750 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bd49db850 con 0x561bd4a80c80 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fbaffd640 10 client.25543 handle_command_reply: tid=0 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fe40f7740 1 client.25543 shutdown 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fe40f7740 1 -- 172.21.15.31:0/1963711750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 msgr2=0x561bd4a70220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a70220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009900 tx=0x7f4fc40044f0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fe40f7740 1 -- 172.21.15.31:0/1963711750 shutdown_connections 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 0x561bd4a70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561bd4a80c80 0x561bd4a830a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a70220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fbaffd640 0 client.25543 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/1963711750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a72f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.271+0000 7f4fe40f7740 1 -- 172.21.15.31:0/1963711750 >> 172.21.15.31:0/1963711750 conn(0x561bd4a64a30 msgr2=0x561bd4a63b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.272+0000 7f4fe40f7740 1 -- 172.21.15.31:0/1963711750 shutdown_connections 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.272+0000 7f4fe40f7740 1 -- 172.21.15.31:0/1963711750 wait complete. 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.272+0000 7f4fe40f7740 20 client.25543 trim_cache size 0 max 16384 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.290+0000 7f4fe40f7740 1 Processor -- start 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.290+0000 7f4fe40f7740 1 -- start start 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.290+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a6d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.290+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a6d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.290+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 0x561bd4a71a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.290+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc1aaa70 con 0x561bd4a684a0 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.290+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc1ad2d0 con 0x561bd4a73680 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.290+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc1ab0c0 con 0x561bd4a68dc0 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.290+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a6d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a6d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55224/0 (socket says 172.21.15.31:55224) 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fdbfff640 1 -- 172.21.15.31:0/2197499851 learned_addr learned my addr 172.21.15.31:0/2197499851 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 0x561bd4a71a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a6d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fdbfff640 1 -- 172.21.15.31:0/2197499851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 msgr2=0x561bd4a6d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a6d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fdbfff640 1 -- 172.21.15.31:0/2197499851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 msgr2=0x561bd4a71a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:04.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 0x561bd4a71a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fdbfff640 1 -- 172.21.15.31:0/2197499851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd4a83720 con 0x561bd4a684a0 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a6d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a6d480 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4011610 tx=0x7f4fd4011640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7783f69b1359a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fd17fa640 1 -- 172.21.15.31:0/2197499851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4025040 con 0x561bd4a684a0 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fd17fa640 1 -- 172.21.15.31:0/2197499851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd4026040 con 0x561bd4a684a0 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.291+0000 7f4fd17fa640 1 -- 172.21.15.31:0/2197499851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd402cce0 con 0x561bd4a684a0 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2197499851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 msgr2=0x561bd4a6d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a6d480 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4011610 tx=0x7f4fd4011640 comp rx=0 tx=0).stop 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2197499851 shutdown_connections 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a73680 0x561bd4a71a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a68dc0 0x561bd4a6d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2197499851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a684a0 0x561bd4a6d480 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2197499851 >> 172.21.15.31:0/2197499851 conn(0x561bd4a64a30 msgr2=0x561bd4a6f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:04.449 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:46:04.449 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:46:04.449 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:46:04.449 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:46:04.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2197499851 shutdown_connections 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2197499851 wait complete. 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.292+0000 7f4fe40f7740 1 -- start start 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fe40f7740 1 Processor -- start 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a684a0 0x561bd4a8ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a68dc0 0x561bd4a8cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fe40f7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a73680 0x561bd4ae6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc1ab790 con 0x561bd4a73680 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc1acc00 con 0x561bd4a68dc0 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fe40f7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc1abe60 con 0x561bd4a684a0 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fe40f7740 10 client.0 _ll_register_callbacks cb 0x7f4fc4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:04.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a73680 0x561bd4ae6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a73680 0x561bd4ae6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55234/0 (socket says 172.21.15.31:55234) 2024-09-16T00:46:04.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fdbfff640 1 -- 172.21.15.31:0/2365566690 learned_addr learned my addr 172.21.15.31:0/2365566690 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:04.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a68dc0 0x561bd4a8cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a684a0 0x561bd4a8ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fd3fff640 1 -- 172.21.15.31:0/2365566690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a684a0 msgr2=0x561bd4a8ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:04.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a684a0 0x561bd4a8ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fd3fff640 1 -- 172.21.15.31:0/2365566690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a73680 msgr2=0x561bd4ae6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:04.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a73680 0x561bd4ae6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fd3fff640 1 -- 172.21.15.31:0/2365566690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd4ae6a90 con 0x561bd4a68dc0 2024-09-16T00:46:04.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.293+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a73680 0x561bd4ae6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.294+0000 7f4fd3fff640 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a68dc0 0x561bd4a8cfd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006010 tx=0x7f4fc4031470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caf37c50c302816f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.294+0000 7f4fbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:46:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.294+0000 7f4fbaffd640 1 -- 172.21.15.31:0/2365566690 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4068060 con 0x561bd4a68dc0 2024-09-16T00:46:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.294+0000 7f4fbaffd640 1 -- 172.21.15.31:0/2365566690 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f4fc40035f0 con 0x561bd4a68dc0 2024-09-16T00:46:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.294+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2365566690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd4026040 con 0x561bd4a68dc0 2024-09-16T00:46:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.294+0000 7f4fbaffd640 1 -- 172.21.15.31:0/2365566690 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4fc406f050 con 0x561bd4a68dc0 2024-09-16T00:46:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.294+0000 7f4fe40f7740 10 client.25610 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.294+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2365566690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561bd4aede50 con 0x561bd4a68dc0 2024-09-16T00:46:04.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.294+0000 7f4fbaffd640 1 -- 172.21.15.31:0/2365566690 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd4026040 con 0x561bd4a68dc0 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fbaffd640 1 -- 172.21.15.31:0/2365566690 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f4fc4054040 con 0x561bd4a68dc0 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fe40f7740 10 client.25610 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fe40f7740 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fe40f7740 20 client.25610 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561bd4aee600 0x561bd4af09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fe40f7740 4 client.25610 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2365566690 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561bd49d5d60 con 0x561bd4aee600 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561bd4aee600 0x561bd4af09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fdbfff640 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561bd4aee600 0x561bd4af09f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5505f0c69dee8ab0 server_cookie=ede41af1fdbd661b in_seq=0 out_seq=0 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.295+0000 7f4fbaffd640 10 client.25610 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fbaffd640 1 -- 172.21.15.31:0/2365566690 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561bd49d5d60 con 0x561bd4aee600 2024-09-16T00:46:04.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fbaffd640 10 client.25610 handle_command_reply: tid=0 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 client.25610 shutdown 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2365566690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a68dc0 msgr2=0x561bd4a8cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a68dc0 0x561bd4a8cfd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006010 tx=0x7f4fc4031470 comp rx=0 tx=0).stop 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2365566690 shutdown_connections 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bd4a73680 0x561bd4ae6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd4a68dc0 0x561bd4a8cfd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561bd4aee600 0x561bd4af09f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 --2- 172.21.15.31:0/2365566690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd4a684a0 0x561bd4a8ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fbaffd640 0 client.25610 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2365566690 >> 172.21.15.31:0/2365566690 conn(0x561bd4a64a30 msgr2=0x561bd4a71a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:04.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2365566690 shutdown_connections 2024-09-16T00:46:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.296+0000 7f4fe40f7740 1 -- 172.21.15.31:0/2365566690 wait complete. 2024-09-16T00:46:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe40f7740 20 client.25610 trim_cache size 0 max 16384 2024-09-16T00:46:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/153227861 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4fc80a32c0 msgr2=0x7f4fc80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4fc80a32c0 0x7f4fc80a5780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc098450 tx=0x7f4fcc086040 comp rx=0 tx=0).stop 2024-09-16T00:46:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/153227861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fdc100a80 msgr2=0x7f4fdc0fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fdc100a80 0x7f4fdc0fce00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00d720 tx=0x7f4fcc00dbe0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/153227861 shutdown_connections 2024-09-16T00:46:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4fc80a32c0 0x7f4fc80a5780 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fdc106a80 0x7f4fdc0fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fdc101420 0x7f4fdc0fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 --2- 172.21.15.31:0/153227861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fdc100a80 0x7f4fdc0fce00 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00d720 tx=0x7f4fcc00dbe0 comp rx=0 tx=0).stop 2024-09-16T00:46:04.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/153227861 >> 172.21.15.31:0/153227861 conn(0x7f4fdc0fc7a0 msgr2=0x7f4fdc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:04.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/153227861 shutdown_connections 2024-09-16T00:46:04.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:04.297+0000 7f4fe1d1e640 1 -- 172.21.15.31:0/153227861 wait complete. 2024-09-16T00:46:05.332 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:46:05.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:46:05.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:05 smithi031 ceph-mon[30956]: pgmap v504: 129 pgs: 129 active+clean; 559 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.5 MiB/s wr, 581 op/s 2024-09-16T00:46:05.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:05 smithi081 ceph-mon[35539]: pgmap v504: 129 pgs: 129 active+clean; 559 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.5 MiB/s wr, 581 op/s 2024-09-16T00:46:06.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:05 smithi102 ceph-mon[35364]: pgmap v504: 129 pgs: 129 active+clean; 559 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.5 MiB/s wr, 581 op/s 2024-09-16T00:46:06.188 INFO:tasks.workunit.client.0.smithi031.stdout: 100 407507 276 264545 186 242751 599 2024-09-16T00:46:07.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:07 smithi031 ceph-mon[30956]: pgmap v505: 129 pgs: 129 active+clean; 562 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 490 op/s 2024-09-16T00:46:07.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:07 smithi081 ceph-mon[35539]: pgmap v505: 129 pgs: 129 active+clean; 562 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 490 op/s 2024-09-16T00:46:08.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:07 smithi102 ceph-mon[35364]: pgmap v505: 129 pgs: 129 active+clean; 562 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 490 op/s 2024-09-16T00:46:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.432+0000 7f89d6229640 1 -- 172.21.15.31:0/3233993032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101470 msgr2=0x7f89d0101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.432+0000 7f89d6229640 1 --2- 172.21.15.31:0/3233993032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101470 0x7f89d0101850 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009900 tx=0x7f89c002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.432+0000 7f89d6229640 1 -- 172.21.15.31:0/3233993032 shutdown_connections 2024-09-16T00:46:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.432+0000 7f89d6229640 1 --2- 172.21.15.31:0/3233993032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89d0105de0 0x7f89d010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.432+0000 7f89d6229640 1 --2- 172.21.15.31:0/3233993032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89d0101e20 0x7f89d01056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.432+0000 7f89d6229640 1 --2- 172.21.15.31:0/3233993032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101470 0x7f89d0101850 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.432+0000 7f89d6229640 1 -- 172.21.15.31:0/3233993032 >> 172.21.15.31:0/3233993032 conn(0x7f89d0077230 msgr2=0x7f89d00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.432+0000 7f89d6229640 1 -- 172.21.15.31:0/3233993032 shutdown_connections 2024-09-16T00:46:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.432+0000 7f89d6229640 1 -- 172.21.15.31:0/3233993032 wait complete. 2024-09-16T00:46:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.433+0000 7f89d6229640 1 Processor -- start 2024-09-16T00:46:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.435+0000 7f89d6229640 1 -- start start 2024-09-16T00:46:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.442+0000 7f89d6229640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89d0101470 0x7f89d0073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.442+0000 7f89d6229640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101e20 0x7f89d006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.442+0000 7f89d6229640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89d0105de0 0x7f89d006c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.442+0000 7f89d6229640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0111f60 con 0x7f89d0101e20 2024-09-16T00:46:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.442+0000 7f89d6229640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0111de0 con 0x7f89d0105de0 2024-09-16T00:46:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.442+0000 7f89d6229640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01120e0 con 0x7f89d0101470 2024-09-16T00:46:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.443+0000 7f89ceffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101e20 0x7f89d006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.443+0000 7f89ceffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101e20 0x7f89d006bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59748/0 (socket says 172.21.15.31:59748) 2024-09-16T00:46:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.443+0000 7f89ceffd640 1 -- 172.21.15.31:0/879698981 learned_addr learned my addr 172.21.15.31:0/879698981 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.444+0000 7f89cf7fe640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89d0101470 0x7f89d0073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.445+0000 7f89ceffd640 1 -- 172.21.15.31:0/879698981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89d0101470 msgr2=0x7f89d0073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.445+0000 7f89ceffd640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89d0101470 0x7f89d0073d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.445+0000 7f89ceffd640 1 -- 172.21.15.31:0/879698981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89d0105de0 msgr2=0x7f89d006c220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:46:09.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.445+0000 7f89ceffd640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89d0105de0 0x7f89d006c220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.445+0000 7f89ceffd640 1 -- 172.21.15.31:0/879698981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d006ca40 con 0x7f89d0101e20 2024-09-16T00:46:09.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.445+0000 7f89ceffd640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101e20 0x7f89d006bce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f89c400e9f0 tx=0x7f89c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.450+0000 7f89ccff9640 1 -- 172.21.15.31:0/879698981 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c400cd80 con 0x7f89d0101e20 2024-09-16T00:46:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.450+0000 7f89ccff9640 1 -- 172.21.15.31:0/879698981 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c4010910 con 0x7f89d0101e20 2024-09-16T00:46:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.451+0000 7f89ccff9640 1 -- 172.21.15.31:0/879698981 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c40188f0 con 0x7f89d0101e20 2024-09-16T00:46:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.451+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d0107910 con 0x7f89d0101e20 2024-09-16T00:46:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.451+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d0107e00 con 0x7f89d0101e20 2024-09-16T00:46:09.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.453+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89bc005290 con 0x7f89d0101e20 2024-09-16T00:46:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.457+0000 7f89ccff9640 1 -- 172.21.15.31:0/879698981 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f89c4010430 con 0x7f89d0101e20 2024-09-16T00:46:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.458+0000 7f89ccff9640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89b80a3410 0x7f89b80a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.458+0000 7f89ccff9640 1 -- 172.21.15.31:0/879698981 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f89c40d02d0 con 0x7f89d0101e20 2024-09-16T00:46:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.458+0000 7f89cf7fe640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89b80a3410 0x7f89b80a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.459+0000 7f89cf7fe640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89b80a3410 0x7f89b80a58d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f89c00096c0 tx=0x7f89c0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.465+0000 7f89ccff9640 1 -- 172.21.15.31:0/879698981 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89c4026080 con 0x7f89d0101e20 2024-09-16T00:46:09.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.742+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89bc005580 con 0x7f89d0101e20 2024-09-16T00:46:09.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.743+0000 7f89ccff9640 1 -- 172.21.15.31:0/879698981 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f89c409cc60 con 0x7f89d0101e20 2024-09-16T00:46:09.743 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:46:09.743 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:46:09.744 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:46:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89b80a3410 msgr2=0x7f89b80a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89b80a3410 0x7f89b80a58d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f89c00096c0 tx=0x7f89c0000bc0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101e20 msgr2=0x7f89d006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101e20 0x7f89d006bce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f89c400e9f0 tx=0x7f89c400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 shutdown_connections 2024-09-16T00:46:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89d0105de0 0x7f89d006c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d0101e20 0x7f89d006bce0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f89b80a3410 0x7f89b80a58d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 --2- 172.21.15.31:0/879698981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89d0101470 0x7f89d0073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:09.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.745+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 >> 172.21.15.31:0/879698981 conn(0x7f89d0077230 msgr2=0x7f89d0104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:09.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.746+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 shutdown_connections 2024-09-16T00:46:09.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:09.746+0000 7f89d6229640 1 -- 172.21.15.31:0/879698981 wait complete. 2024-09-16T00:46:09.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:09 smithi031 ceph-mon[30956]: pgmap v506: 129 pgs: 129 active+clean; 553 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 567 op/s 2024-09-16T00:46:09.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:09 smithi081 ceph-mon[35539]: pgmap v506: 129 pgs: 129 active+clean; 553 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 567 op/s 2024-09-16T00:46:10.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:09 smithi102 ceph-mon[35364]: pgmap v506: 129 pgs: 129 active+clean; 553 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 567 op/s 2024-09-16T00:46:10.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:10 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/879698981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:10.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:10 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/879698981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:11.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:10 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/879698981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:11.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:46:11.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:11 smithi031 ceph-mon[30956]: pgmap v507: 129 pgs: 129 active+clean; 540 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 477 op/s 2024-09-16T00:46:11.950 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:11 smithi102 ceph-mon[35364]: pgmap v507: 129 pgs: 129 active+clean; 540 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 477 op/s 2024-09-16T00:46:11.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:11 smithi081 ceph-mon[35539]: pgmap v507: 129 pgs: 129 active+clean; 540 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 477 op/s 2024-09-16T00:46:13.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:13 smithi031 ceph-mon[30956]: pgmap v508: 129 pgs: 129 active+clean; 535 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 450 op/s 2024-09-16T00:46:13.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:13 smithi081 ceph-mon[35539]: pgmap v508: 129 pgs: 129 active+clean; 535 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 450 op/s 2024-09-16T00:46:14.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:13 smithi102 ceph-mon[35364]: pgmap v508: 129 pgs: 129 active+clean; 535 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 450 op/s 2024-09-16T00:46:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:14.631 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:46:14.631 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-16T00:46:14.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:46:15.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.273+0000 7fcfe4e34640 1 -- 172.21.15.31:0/2700127119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe010d520 msgr2=0x7fcfe01150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:15.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.273+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/2700127119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe010d520 0x7fcfe01150a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009be0 tx=0x7fcfd802a710 comp rx=0 tx=0).stop 2024-09-16T00:46:15.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.275+0000 7fcfe4e34640 1 -- 172.21.15.31:0/2700127119 shutdown_connections 2024-09-16T00:46:15.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.275+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/2700127119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe010d520 0x7fcfe01150a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.275+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/2700127119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfe0108da0 0x7fcfe010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.275+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/2700127119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe01083f0 0x7fcfe01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.275+0000 7fcfe4e34640 1 -- 172.21.15.31:0/2700127119 >> 172.21.15.31:0/2700127119 conn(0x7fcfe0077af0 msgr2=0x7fcfe0079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:15.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.279+0000 7fcfe4e34640 1 -- 172.21.15.31:0/2700127119 shutdown_connections 2024-09-16T00:46:15.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.280+0000 7fcfe4e34640 1 -- 172.21.15.31:0/2700127119 wait complete. 2024-09-16T00:46:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.281+0000 7fcfe4e34640 1 Processor -- start 2024-09-16T00:46:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.281+0000 7fcfe4e34640 1 -- start start 2024-09-16T00:46:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.281+0000 7fcfe4e34640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe01083f0 0x7fcfe00827d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.281+0000 7fcfe4e34640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfe0108da0 0x7fcfe0082d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.281+0000 7fcfe4e34640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe010d520 0x7fcfe007ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.281+0000 7fcfe4e34640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0118550 con 0x7fcfe0108da0 2024-09-16T00:46:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.281+0000 7fcfe4e34640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01183d0 con 0x7fcfe010d520 2024-09-16T00:46:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.281+0000 7fcfe4e34640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01186d0 con 0x7fcfe01083f0 2024-09-16T00:46:15.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.288+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfe0108da0 0x7fcfe0082d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.288+0000 7fcfde59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe01083f0 0x7fcfe00827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.288+0000 7fcfde59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe01083f0 0x7fcfe00827d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:44118/0 (socket says 172.21.15.31:44118) 2024-09-16T00:46:15.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.288+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfe0108da0 0x7fcfe0082d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59766/0 (socket says 172.21.15.31:59766) 2024-09-16T00:46:15.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.288+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1846408443 learned_addr learned my addr 172.21.15.31:0/1846408443 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:15.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.288+0000 7fcfded9d640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe010d520 0x7fcfe007ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.289+0000 7fcfde59c640 1 -- 172.21.15.31:0/1846408443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe010d520 msgr2=0x7fcfe007ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.289+0000 7fcfde59c640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe010d520 0x7fcfe007ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.289+0000 7fcfde59c640 1 -- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfe0108da0 msgr2=0x7fcfe0082d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.289+0000 7fcfde59c640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfe0108da0 0x7fcfe0082d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.289+0000 7fcfde59c640 1 -- 172.21.15.31:0/1846408443 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe007c300 con 0x7fcfe01083f0 2024-09-16T00:46:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.289+0000 7fcfd7fff640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfe0108da0 0x7fcfe0082d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.291+0000 7fcfde59c640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe01083f0 0x7fcfe00827d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000e9f0 tx=0x7fcfd000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:15.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.297+0000 7fcfd77fe640 1 -- 172.21.15.31:0/1846408443 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000ccc0 con 0x7fcfe01083f0 2024-09-16T00:46:15.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.297+0000 7fcfd77fe640 1 -- 172.21.15.31:0/1846408443 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd0010910 con 0x7fcfe01083f0 2024-09-16T00:46:15.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.297+0000 7fcfd77fe640 1 -- 172.21.15.31:0/1846408443 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0018800 con 0x7fcfe01083f0 2024-09-16T00:46:15.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.297+0000 7fcfe4e34640 1 -- 172.21.15.31:0/1846408443 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe007c5f0 con 0x7fcfe01083f0 2024-09-16T00:46:15.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.298+0000 7fcfe4e34640 1 -- 172.21.15.31:0/1846408443 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe0080590 con 0x7fcfe01083f0 2024-09-16T00:46:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.306+0000 7fcfe720d740 1 Processor -- start 2024-09-16T00:46:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.306+0000 7fcfe720d740 1 -- start start 2024-09-16T00:46:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.306+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad062090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.307+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad062a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.307+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.307+0000 7fcfe720d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5555acfc82f0 con 0x5555ad063b40 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.307+0000 7fcfe720d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5555acf6fcf0 con 0x5555ad0625d0 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.307+0000 7fcfe720d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5555acfc42b0 con 0x5555ad06d370 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.307+0000 7fcfded9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad062090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.307+0000 7fcfded9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad062090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59778/0 (socket says 172.21.15.31:59778) 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.307+0000 7fcfded9d640 1 -- 172.21.15.31:0/1265655667 learned_addr learned my addr 172.21.15.31:0/1265655667 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.307+0000 7fcfd7fff640 1 --2- 172.21.15.31:0/1265655667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.308+0000 7fcfded9d640 1 -- 172.21.15.31:0/1265655667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 msgr2=0x5555ad06f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.308+0000 7fcfded9d640 1 --2- 172.21.15.31:0/1265655667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.308+0000 7fcfded9d640 1 -- 172.21.15.31:0/1265655667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 msgr2=0x5555ad062a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.309+0000 7fcfded9d640 1 --2- 172.21.15.31:0/1265655667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad062a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.309+0000 7fcfded9d640 1 -- 172.21.15.31:0/1265655667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555acf543a0 con 0x5555ad063b40 2024-09-16T00:46:15.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.309+0000 7fcfded9d640 1 --2- 172.21.15.31:0/1265655667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad062090 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8033000 tx=0x7fcfd80304a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8bf7568c0781db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:15.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.312+0000 7fcfd4ff9640 1 -- 172.21.15.31:0/1265655667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8064070 con 0x5555ad063b40 2024-09-16T00:46:15.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.312+0000 7fcfd4ff9640 1 -- 172.21.15.31:0/1265655667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd8030e60 con 0x5555ad063b40 2024-09-16T00:46:15.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.312+0000 7fcfd4ff9640 1 -- 172.21.15.31:0/1265655667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8036040 con 0x5555ad063b40 2024-09-16T00:46:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.316+0000 7fcfe720d740 1 -- 172.21.15.31:0/1265655667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 msgr2=0x5555ad062090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.316+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1265655667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad062090 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8033000 tx=0x7fcfd80304a0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.319+0000 7fcfd77fe640 1 -- 172.21.15.31:0/1846408443 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcfd0010ac0 con 0x7fcfe01083f0 2024-09-16T00:46:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.320+0000 7fcfd77fe640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcfcc0a3450 0x7fcfcc0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.321+0000 7fcfd77fe640 1 -- 172.21.15.31:0/1846408443 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fcfd00d00b0 con 0x7fcfe01083f0 2024-09-16T00:46:15.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.321+0000 7fcfe720d740 1 -- 172.21.15.31:0/1265655667 shutdown_connections 2024-09-16T00:46:15.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.321+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1265655667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.321+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1265655667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad062a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.321+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1265655667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad062090 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.321+0000 7fcfe720d740 1 -- 172.21.15.31:0/1265655667 >> 172.21.15.31:0/1265655667 conn(0x5555ad02b310 msgr2=0x5555ad05ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:15.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.324+0000 7fcfe720d740 1 -- 172.21.15.31:0/1265655667 shutdown_connections 2024-09-16T00:46:15.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.326+0000 7fcfded9d640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcfcc0a3450 0x7fcfcc0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.326+0000 7fcfe720d740 1 -- 172.21.15.31:0/1265655667 wait complete. 2024-09-16T00:46:15.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.327+0000 7fcfe720d740 1 -- start start 2024-09-16T00:46:15.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.327+0000 7fcfe720d740 1 Processor -- start 2024-09-16T00:46:15.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.329+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad06cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.329+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad06a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.329+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.329+0000 7fcfe720d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5555ad070010 con 0x5555ad063b40 2024-09-16T00:46:15.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.329+0000 7fcfe720d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5555acf83060 con 0x5555ad0625d0 2024-09-16T00:46:15.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.329+0000 7fcfe720d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5555ad070190 con 0x5555ad06d370 2024-09-16T00:46:15.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.329+0000 7fcfe720d740 10 client.0 _ll_register_callbacks cb 0x5555accd68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:15.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.330+0000 7fcfded9d640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcfcc0a3450 0x7fcfcc0a5910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcfd802ad00 tx=0x7fcfd802a710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:15.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.330+0000 7fcfded9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.330+0000 7fcfded9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:44128/0 (socket says 172.21.15.31:44128) 2024-09-16T00:46:15.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.330+0000 7fcfded9d640 1 -- 172.21.15.31:0/539140350 learned_addr learned my addr 172.21.15.31:0/539140350 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:15.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.331+0000 7fcfde59c640 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad06cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.331+0000 7fcfded9d640 1 -- 172.21.15.31:0/539140350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 msgr2=0x5555ad06cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:15.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.331+0000 7fcfded9d640 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad06cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.331+0000 7fcfded9d640 1 -- 172.21.15.31:0/539140350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 msgr2=0x5555ad06a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:15.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.331+0000 7fcfded9d640 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad06a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.331+0000 7fcfded9d640 1 -- 172.21.15.31:0/539140350 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555acd393c0 con 0x5555ad06d370 2024-09-16T00:46:15.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.331+0000 7fcfde59c640 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad06cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:46:15.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.332+0000 7fcfded9d640 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06a840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8066000 tx=0x7fcfd802ac10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d095c973f165fdfa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:15.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.332+0000 7fcfc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:46:15.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.333+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/539140350 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8036040 con 0x5555ad06d370 2024-09-16T00:46:15.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.333+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/539140350 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fcfd805f670 con 0x5555ad06d370 2024-09-16T00:46:15.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.334+0000 7fcfe720d740 1 -- 172.21.15.31:0/539140350 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd8030e60 con 0x5555ad06d370 2024-09-16T00:46:15.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.334+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/539140350 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcfd8064040 con 0x5555ad06d370 2024-09-16T00:46:15.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.334+0000 7fcfe720d740 10 client.25564 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:15.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.334+0000 7fcfe720d740 1 -- 172.21.15.31:0/539140350 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5555ad070010 con 0x5555ad06d370 2024-09-16T00:46:15.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.335+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/539140350 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd8030e60 con 0x5555ad06d370 2024-09-16T00:46:15.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.335+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/539140350 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fcfd8069890 con 0x5555ad06d370 2024-09-16T00:46:15.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.335+0000 7fcfe720d740 10 client.25564 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:15.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.335+0000 7fcfe720d740 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:46:15.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.335+0000 7fcfe720d740 20 client.25564 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:15.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.335+0000 7fcfe720d740 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5555ad07d140 0x5555ad07f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.335+0000 7fcfe720d740 4 client.25564 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:46:15.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.335+0000 7fcfe720d740 1 -- 172.21.15.31:0/539140350 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555acfd5860 con 0x5555ad07d140 2024-09-16T00:46:15.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.338+0000 7fcfde59c640 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5555ad07d140 0x5555ad07f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.339 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:15.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.338+0000 7fcfde59c640 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5555ad07d140 0x5555ad07f5a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8871d033ae72e0e3 server_cookie=3326176146fa5e73 in_seq=0 out_seq=0 2024-09-16T00:46:15.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.339+0000 7fcfc5ffb640 10 client.25564 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:15.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.342+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/539140350 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcfd00abe80 con 0x5555ad07d140 2024-09-16T00:46:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.342+0000 7fcfc5ffb640 10 client.25564 handle_command_reply: tid=0 2024-09-16T00:46:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.342+0000 7fcfe720d740 1 client.25564 shutdown 2024-09-16T00:46:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.343+0000 7fcfe720d740 1 -- 172.21.15.31:0/539140350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 msgr2=0x5555ad06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:15.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.343+0000 7fcfe720d740 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06a840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8066000 tx=0x7fcfd802ac10 comp rx=0 tx=0).stop 2024-09-16T00:46:15.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.343+0000 7fcfe720d740 1 -- 172.21.15.31:0/539140350 shutdown_connections 2024-09-16T00:46:15.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.343+0000 7fcfe720d740 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5555ad07d140 0x5555ad07f5a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.343+0000 7fcfe720d740 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad06d370 0x5555ad06a840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.343+0000 7fcfe720d740 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad06a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.343+0000 7fcfe720d740 1 --2- 172.21.15.31:0/539140350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad06cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.343+0000 7fcfe720d740 1 -- 172.21.15.31:0/539140350 >> 172.21.15.31:0/539140350 conn(0x5555ad02b310 msgr2=0x5555ad064930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.344+0000 7fcfc5ffb640 0 client.25564 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.345+0000 7fcfe720d740 1 -- 172.21.15.31:0/539140350 shutdown_connections 2024-09-16T00:46:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.345+0000 7fcfe720d740 1 -- 172.21.15.31:0/539140350 wait complete. 2024-09-16T00:46:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.345+0000 7fcfe720d740 20 client.25564 trim_cache size 0 max 16384 2024-09-16T00:46:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.362+0000 7fcfe720d740 1 Processor -- start 2024-09-16T00:46:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.362+0000 7fcfe720d740 1 -- start start 2024-09-16T00:46:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.363+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad0625d0 0x5555ad065120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.363+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad065660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.363+0000 7fcfde59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad0625d0 0x5555ad065120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.363+0000 7fcfde59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad0625d0 0x5555ad065120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:44140/0 (socket says 172.21.15.31:44140) 2024-09-16T00:46:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.363+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad065660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.363+0000 7fcfde59c640 1 -- 172.21.15.31:0/101412334 learned_addr learned my addr 172.21.15.31:0/101412334 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.363+0000 7fcfe720d740 1 --2- 172.21.15.31:0/101412334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad06d370 0x5555ad084eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.364+0000 7fcfded9d640 1 --2- 172.21.15.31:0/101412334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad06d370 0x5555ad084eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.364+0000 7fcfe720d740 1 -- 172.21.15.31:0/101412334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01087d0 con 0x5555ad063b40 2024-09-16T00:46:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.364+0000 7fcfe720d740 1 -- 172.21.15.31:0/101412334 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0119f30 con 0x5555ad06d370 2024-09-16T00:46:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.364+0000 7fcfe720d740 1 -- 172.21.15.31:0/101412334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0073470 con 0x5555ad0625d0 2024-09-16T00:46:15.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.365+0000 7fcfd7fff640 1 -- 172.21.15.31:0/101412334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad0625d0 msgr2=0x5555ad065120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:15.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.365+0000 7fcfd7fff640 1 --2- 172.21.15.31:0/101412334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad0625d0 0x5555ad065120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.365+0000 7fcfd7fff640 1 -- 172.21.15.31:0/101412334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad06d370 msgr2=0x5555ad084eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:15.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.365+0000 7fcfd7fff640 1 --2- 172.21.15.31:0/101412334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad06d370 0x5555ad084eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.365+0000 7fcfd7fff640 1 -- 172.21.15.31:0/101412334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ad068620 con 0x5555ad063b40 2024-09-16T00:46:15.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.365+0000 7fcfd7fff640 1 --2- 172.21.15.31:0/101412334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad065660 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5555ad06b680 tx=0x7fcfc80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755f516a3438c8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:15.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.366+0000 7fcfc7fff640 1 -- 172.21.15.31:0/101412334 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8004050 con 0x5555ad063b40 2024-09-16T00:46:15.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.366+0000 7fcfc7fff640 1 -- 172.21.15.31:0/101412334 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8005500 con 0x5555ad063b40 2024-09-16T00:46:15.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.366+0000 7fcfc7fff640 1 -- 172.21.15.31:0/101412334 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80035f0 con 0x5555ad063b40 2024-09-16T00:46:15.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.367+0000 7fcfe720d740 1 -- 172.21.15.31:0/101412334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 msgr2=0x5555ad065660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:15.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.367+0000 7fcfe720d740 1 --2- 172.21.15.31:0/101412334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad065660 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5555ad06b680 tx=0x7fcfc80118d0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.367+0000 7fcfe720d740 1 -- 172.21.15.31:0/101412334 shutdown_connections 2024-09-16T00:46:15.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.367+0000 7fcfe720d740 1 --2- 172.21.15.31:0/101412334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad06d370 0x5555ad084eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.367+0000 7fcfe720d740 1 --2- 172.21.15.31:0/101412334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad063b40 0x5555ad065660 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.367+0000 7fcfe720d740 1 --2- 172.21.15.31:0/101412334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad0625d0 0x5555ad065120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.367+0000 7fcfe720d740 1 -- 172.21.15.31:0/101412334 >> 172.21.15.31:0/101412334 conn(0x5555ad02b310 msgr2=0x5555ad0691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.367+0000 7fcfe720d740 1 -- 172.21.15.31:0/101412334 shutdown_connections 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.368+0000 7fcfe720d740 1 -- 172.21.15.31:0/101412334 wait complete. 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.368+0000 7fcfe720d740 1 -- start start 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.369+0000 7fcfe720d740 1 Processor -- start 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.369+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad075880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.370+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad06d370 0x5555ad066a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.370+0000 7fcfe720d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad066f60 0x5555ad0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.370+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad06d370 0x5555ad066a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.370+0000 7fcfe720d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0073b00 con 0x5555ad06d370 2024-09-16T00:46:15.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.370+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad06d370 0x5555ad066a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59794/0 (socket says 172.21.15.31:59794) 2024-09-16T00:46:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.370+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1086997007 learned_addr learned my addr 172.21.15.31:0/1086997007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.370+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0119860 con 0x5555ad0625d0 2024-09-16T00:46:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.370+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe00741d0 con 0x5555ad066f60 2024-09-16T00:46:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.371+0000 7fcfe720d740 10 client.0 _ll_register_callbacks cb 0x7fcfc8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.371+0000 7fcfde59c640 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad075880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.371+0000 7fcfded9d640 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad066f60 0x5555ad0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.371+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1086997007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad066f60 msgr2=0x5555ad0673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.371+0000 7fcfd7fff640 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad066f60 0x5555ad0673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.371+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1086997007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 msgr2=0x5555ad075880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.371+0000 7fcfd7fff640 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad075880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.371+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1086997007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ad0e6e80 con 0x5555ad06d370 2024-09-16T00:46:15.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.372+0000 7fcfd7fff640 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad06d370 0x5555ad066a20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8011fd0 tx=0x7fcfc8004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f920cf43d2cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:15.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.373+0000 7fcfc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:46:15.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.373+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/1086997007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80056b0 con 0x5555ad06d370 2024-09-16T00:46:15.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.373+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc8005500 con 0x5555ad06d370 2024-09-16T00:46:15.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.373+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/1086997007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fcfc8005230 con 0x5555ad06d370 2024-09-16T00:46:15.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.374+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/1086997007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcfc8031050 con 0x5555ad06d370 2024-09-16T00:46:15.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.374+0000 7fcfe720d740 10 client.15960 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.374+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5555ad0e7fa0 con 0x5555ad06d370 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.374+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/1086997007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8005500 con 0x5555ad06d370 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.376+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/1086997007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fcfc8003c70 con 0x5555ad06d370 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.376+0000 7fcfe720d740 10 client.15960 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.376+0000 7fcfe720d740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.376+0000 7fcfe720d740 20 client.15960 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.376+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5555ad0e85e0 0x5555ad0eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.376+0000 7fcfe720d740 4 client.15960 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.376+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555acfcfd70 con 0x5555ad0e85e0 2024-09-16T00:46:15.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.376+0000 7fcfde59c640 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5555ad0e85e0 0x5555ad0eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:15.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.377+0000 7fcfde59c640 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5555ad0e85e0 0x5555ad0eaa00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3741f094b72c860d server_cookie=10f778fb8a740b8 in_seq=0 out_seq=0 2024-09-16T00:46:15.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.377+0000 7fcfc5ffb640 10 client.15960 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:15.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.379+0000 7fcfc5ffb640 1 -- 172.21.15.31:0/1086997007 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555acfcfd70 con 0x5555ad0e85e0 2024-09-16T00:46:15.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.380+0000 7fcfc5ffb640 10 client.15960 handle_command_reply: tid=0 2024-09-16T00:46:15.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.380+0000 7fcfe720d740 1 client.15960 shutdown 2024-09-16T00:46:15.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.380+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad06d370 msgr2=0x5555ad066a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:15.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.380+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad06d370 0x5555ad066a20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8011fd0 tx=0x7fcfc8004dd0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 shutdown_connections 2024-09-16T00:46:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5555ad0e85e0 0x5555ad0eaa00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555ad066f60 0x5555ad0673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555ad06d370 0x5555ad066a20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfe720d740 1 --2- 172.21.15.31:0/1086997007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555ad0625d0 0x5555ad075880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 >> 172.21.15.31:0/1086997007 conn(0x5555ad02b310 msgr2=0x5555ad0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfc5ffb640 0 client.15960 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 shutdown_connections 2024-09-16T00:46:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfe720d740 1 -- 172.21.15.31:0/1086997007 wait complete. 2024-09-16T00:46:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.381+0000 7fcfe720d740 20 client.15960 trim_cache size 0 max 16384 2024-09-16T00:46:15.383 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:46:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 -- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcfcc0a3450 msgr2=0x7fcfcc0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcfcc0a3450 0x7fcfcc0a5910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcfd802ad00 tx=0x7fcfd802a710 comp rx=0 tx=0).stop 2024-09-16T00:46:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 -- 172.21.15.31:0/1846408443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe01083f0 msgr2=0x7fcfe00827d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe01083f0 0x7fcfe00827d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000e9f0 tx=0x7fcfd000edf0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 -- 172.21.15.31:0/1846408443 shutdown_connections 2024-09-16T00:46:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe010d520 0x7fcfe007ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfe0108da0 0x7fcfe0082d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcfcc0a3450 0x7fcfcc0a5910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 --2- 172.21.15.31:0/1846408443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfe01083f0 0x7fcfe00827d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:15.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 -- 172.21.15.31:0/1846408443 >> 172.21.15.31:0/1846408443 conn(0x7fcfe0077af0 msgr2=0x7fcfe0078e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:15.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.383+0000 7fcfe4e34640 1 -- 172.21.15.31:0/1846408443 shutdown_connections 2024-09-16T00:46:15.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:15.384+0000 7fcfe4e34640 1 -- 172.21.15.31:0/1846408443 wait complete. 2024-09-16T00:46:15.737 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:15 smithi031 ceph-mon[30956]: pgmap v509: 129 pgs: 129 active+clean; 549 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 441 op/s 2024-09-16T00:46:15.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:15 smithi081 ceph-mon[35539]: pgmap v509: 129 pgs: 129 active+clean; 549 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 441 op/s 2024-09-16T00:46:16.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:15 smithi102 ceph-mon[35364]: pgmap v509: 129 pgs: 129 active+clean; 549 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 441 op/s 2024-09-16T00:46:16.188 INFO:tasks.workunit.client.0.smithi031.stdout: 105 1645836 348 1067218 450 977558 1120 2024-09-16T00:46:16.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.249+0000 7efe6b117640 1 -- 172.21.15.31:0/1289618663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe64101270 msgr2=0x7efe6410d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.250+0000 7efe6b117640 1 --2- 172.21.15.31:0/1289618663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe64101270 0x7efe6410d7a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe60009930 tx=0x7efe6002a480 comp rx=0 tx=0).stop 2024-09-16T00:46:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.250+0000 7efe6b117640 1 -- 172.21.15.31:0/1289618663 shutdown_connections 2024-09-16T00:46:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.250+0000 7efe6b117640 1 --2- 172.21.15.31:0/1289618663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe64101270 0x7efe6410d7a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.250+0000 7efe6b117640 1 --2- 172.21.15.31:0/1289618663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe641008d0 0x7efe64100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.250+0000 7efe6b117640 1 --2- 172.21.15.31:0/1289618663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641068d0 0x7efe64106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.250+0000 7efe6b117640 1 -- 172.21.15.31:0/1289618663 >> 172.21.15.31:0/1289618663 conn(0x7efe640fc610 msgr2=0x7efe640fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:16.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.258+0000 7efe6b117640 1 -- 172.21.15.31:0/1289618663 shutdown_connections 2024-09-16T00:46:16.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.260+0000 7efe6b117640 1 -- 172.21.15.31:0/1289618663 wait complete. 2024-09-16T00:46:16.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.261+0000 7efe6b117640 1 Processor -- start 2024-09-16T00:46:16.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.261+0000 7efe6b117640 1 -- start start 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.261+0000 7efe6b117640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe641008d0 0x7efe64073d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.261+0000 7efe6b117640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe64101270 0x7efe640718a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.261+0000 7efe6b117640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641068d0 0x7efe64071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.261+0000 7efe6b117640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efe6406c660 con 0x7efe64101270 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.261+0000 7efe6b117640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efe6406c4e0 con 0x7efe641008d0 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.261+0000 7efe6b117640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efe6406c7e0 con 0x7efe641068d0 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.262+0000 7efe6a916640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641068d0 0x7efe64071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.262+0000 7efe6a916640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641068d0 0x7efe64071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:44158/0 (socket says 172.21.15.31:44158) 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.262+0000 7efe6a916640 1 -- 172.21.15.31:0/66063348 learned_addr learned my addr 172.21.15.31:0/66063348 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.262+0000 7efe69914640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe64101270 0x7efe640718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.263+0000 7efe69914640 1 -- 172.21.15.31:0/66063348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641068d0 msgr2=0x7efe64071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:16.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.263+0000 7efe69914640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641068d0 0x7efe64071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.263+0000 7efe69914640 1 -- 172.21.15.31:0/66063348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe641008d0 msgr2=0x7efe64073d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:46:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.263+0000 7efe69914640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe641008d0 0x7efe64073d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.263+0000 7efe69914640 1 -- 172.21.15.31:0/66063348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe64072450 con 0x7efe64101270 2024-09-16T00:46:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.264+0000 7efe69914640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe64101270 0x7efe640718a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00eb00 tx=0x7efe5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.265+0000 7efe5b7fe640 1 -- 172.21.15.31:0/66063348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c010430 con 0x7efe64101270 2024-09-16T00:46:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.265+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6406ce20 con 0x7efe64101270 2024-09-16T00:46:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.266+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6406d360 con 0x7efe64101270 2024-09-16T00:46:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.268+0000 7efe5b7fe640 1 -- 172.21.15.31:0/66063348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe5c010ab0 con 0x7efe64101270 2024-09-16T00:46:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.268+0000 7efe5b7fe640 1 -- 172.21.15.31:0/66063348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c017c10 con 0x7efe64101270 2024-09-16T00:46:16.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.272+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe640726e0 con 0x7efe64101270 2024-09-16T00:46:16.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.287+0000 7efe5b7fe640 1 -- 172.21.15.31:0/66063348 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7efe5c010c60 con 0x7efe64101270 2024-09-16T00:46:16.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.289+0000 7efe5b7fe640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efe500a3450 0x7efe500a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:16.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.289+0000 7efe5b7fe640 1 -- 172.21.15.31:0/66063348 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7efe5c0d0c50 con 0x7efe64101270 2024-09-16T00:46:16.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.289+0000 7efe5b7fe640 1 -- 172.21.15.31:0/66063348 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe5c01d060 con 0x7efe64101270 2024-09-16T00:46:16.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.291+0000 7efe6a115640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efe500a3450 0x7efe500a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:16.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.292+0000 7efe6a115640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efe500a3450 0x7efe500a5910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efe54009670 tx=0x7efe54006bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.771+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe64102010 con 0x7efe64101270 2024-09-16T00:46:16.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.776+0000 7efe5b7fe640 1 -- 172.21.15.31:0/66063348 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7efe5c09d5e0 con 0x7efe64101270 2024-09-16T00:46:16.776 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:46:16.777 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:46:16.777 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:46:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.784+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efe500a3450 msgr2=0x7efe500a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.784+0000 7efe6b117640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efe500a3450 0x7efe500a5910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efe54009670 tx=0x7efe54006bd0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.784+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe64101270 msgr2=0x7efe640718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.784+0000 7efe6b117640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe64101270 0x7efe640718a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00eb00 tx=0x7efe5c00c2d0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.789+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 shutdown_connections 2024-09-16T00:46:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.789+0000 7efe6b117640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7efe500a3450 0x7efe500a5910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.789+0000 7efe6b117640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641068d0 0x7efe64071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.789+0000 7efe6b117640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe64101270 0x7efe640718a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.789+0000 7efe6b117640 1 --2- 172.21.15.31:0/66063348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe641008d0 0x7efe64073d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.789+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 >> 172.21.15.31:0/66063348 conn(0x7efe640fc610 msgr2=0x7efe6406a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:16.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.795+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 shutdown_connections 2024-09-16T00:46:16.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:16.795+0000 7efe6b117640 1 -- 172.21.15.31:0/66063348 wait complete. 2024-09-16T00:46:17.875 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:17 smithi031 ceph-mon[30956]: pgmap v510: 129 pgs: 129 active+clean; 558 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 365 op/s 2024-09-16T00:46:17.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:17 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/66063348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:17 smithi081 ceph-mon[35539]: pgmap v510: 129 pgs: 129 active+clean; 558 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 365 op/s 2024-09-16T00:46:17.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:17 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/66063348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:18.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:17 smithi102 ceph-mon[35364]: pgmap v510: 129 pgs: 129 active+clean; 558 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 365 op/s 2024-09-16T00:46:18.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:17 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/66063348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:18.317 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-16T00:46:18.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-16T00:46:18.531 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:46:18.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:46:19.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:19 smithi081 ceph-mon[35539]: pgmap v511: 129 pgs: 129 active+clean; 572 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 441 op/s 2024-09-16T00:46:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:19 smithi102 ceph-mon[35364]: pgmap v511: 129 pgs: 129 active+clean; 572 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 441 op/s 2024-09-16T00:46:20.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:19 smithi031 ceph-mon[30956]: pgmap v511: 129 pgs: 129 active+clean; 572 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 441 op/s 2024-09-16T00:46:21.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:21 smithi081 ceph-mon[35539]: pgmap v512: 129 pgs: 129 active+clean; 583 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 417 op/s 2024-09-16T00:46:22.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:21 smithi102 ceph-mon[35364]: pgmap v512: 129 pgs: 129 active+clean; 583 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 417 op/s 2024-09-16T00:46:22.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:21 smithi031 ceph-mon[30956]: pgmap v512: 129 pgs: 129 active+clean; 583 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 417 op/s 2024-09-16T00:46:22.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:22 smithi081 ceph-mon[35539]: pgmap v513: 129 pgs: 129 active+clean; 582 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 397 op/s 2024-09-16T00:46:23.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:22 smithi102 ceph-mon[35364]: pgmap v513: 129 pgs: 129 active+clean; 582 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 397 op/s 2024-09-16T00:46:23.049 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:23.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:22 smithi031 ceph-mon[30956]: pgmap v513: 129 pgs: 129 active+clean; 582 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 397 op/s 2024-09-16T00:46:24.216 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:24.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.297+0000 7f163ba55640 1 -- 172.21.15.31:0/3553430274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163411b140 msgr2=0x7f163411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:24.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.297+0000 7f163ba55640 1 --2- 172.21.15.31:0/3553430274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163411b140 0x7f163411d530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f162c009ca0 tx=0x7f162c02a8b0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.300+0000 7f163ba55640 1 -- 172.21.15.31:0/3553430274 shutdown_connections 2024-09-16T00:46:24.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.300+0000 7f163ba55640 1 --2- 172.21.15.31:0/3553430274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163411b140 0x7f163411d530 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.300+0000 7f163ba55640 1 --2- 172.21.15.31:0/3553430274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1634109830 0x7f1634109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.300+0000 7f163ba55640 1 --2- 172.21.15.31:0/3553430274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1634109450 0x7f1634110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.300+0000 7f163ba55640 1 -- 172.21.15.31:0/3553430274 >> 172.21.15.31:0/3553430274 conn(0x7f163406e140 msgr2=0x7f163406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:24.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.301+0000 7f163ba55640 1 -- 172.21.15.31:0/3553430274 shutdown_connections 2024-09-16T00:46:24.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.304+0000 7f163ba55640 1 -- 172.21.15.31:0/3553430274 wait complete. 2024-09-16T00:46:24.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.305+0000 7f163ba55640 1 Processor -- start 2024-09-16T00:46:24.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.305+0000 7f163ba55640 1 -- start start 2024-09-16T00:46:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.309+0000 7f163ba55640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1634109450 0x7f16341a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.309+0000 7f163ba55640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1634109830 0x7f16341a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.309+0000 7f163ba55640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f163411b140 0x7f16341aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.309+0000 7f163ba55640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1634120da0 con 0x7f1634109450 2024-09-16T00:46:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.309+0000 7f163ba55640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1634120c20 con 0x7f1634109830 2024-09-16T00:46:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.309+0000 7f163ba55640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1634120f20 con 0x7f163411b140 2024-09-16T00:46:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f1638ff0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1634109830 0x7f16341a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f16397f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1634109450 0x7f16341a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f1638ff0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1634109830 0x7f16341a76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:43530/0 (socket says 172.21.15.31:43530) 2024-09-16T00:46:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f1638ff0640 1 -- 172.21.15.31:0/1609056605 learned_addr learned my addr 172.21.15.31:0/1609056605 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f1639ff2640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f163411b140 0x7f16341aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f1638ff0640 1 -- 172.21.15.31:0/1609056605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f163411b140 msgr2=0x7f16341aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f1638ff0640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f163411b140 0x7f16341aba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f1638ff0640 1 -- 172.21.15.31:0/1609056605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1634109450 msgr2=0x7f16341a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f1638ff0640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1634109450 0x7f16341a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.310+0000 7f1638ff0640 1 -- 172.21.15.31:0/1609056605 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16341ac110 con 0x7f1634109830 2024-09-16T00:46:24.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.316+0000 7f1638ff0640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1634109830 0x7f16341a76a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f162400e9f0 tx=0x7f162400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:24.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.317+0000 7f162a7fc640 1 -- 172.21.15.31:0/1609056605 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162400ccc0 con 0x7f1634109830 2024-09-16T00:46:24.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.317+0000 7f163ba55640 1 -- 172.21.15.31:0/1609056605 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16341ac400 con 0x7f1634109830 2024-09-16T00:46:24.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.317+0000 7f163ba55640 1 -- 172.21.15.31:0/1609056605 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16341b3ce0 con 0x7f1634109830 2024-09-16T00:46:24.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.321+0000 7f163de2e740 1 Processor -- start 2024-09-16T00:46:24.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.321+0000 7f162a7fc640 1 -- 172.21.15.31:0/1609056605 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1624010910 con 0x7f1634109830 2024-09-16T00:46:24.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.321+0000 7f162a7fc640 1 -- 172.21.15.31:0/1609056605 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240188a0 con 0x7f1634109830 2024-09-16T00:46:24.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.321+0000 7f163de2e740 1 -- start start 2024-09-16T00:46:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.325+0000 7f163de2e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d229f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.325+0000 7f163de2e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 0x55655d234800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.325+0000 7f1639ff2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d229f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.325+0000 7f1639ff2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d229f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:43534/0 (socket says 172.21.15.31:43534) 2024-09-16T00:46:24.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.326+0000 7f1639ff2640 1 -- 172.21.15.31:0/2826727963 learned_addr learned my addr 172.21.15.31:0/2826727963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:24.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.326+0000 7f163de2e740 1 --2- 172.21.15.31:0/2826727963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 0x55655d237130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.326+0000 7f163de2e740 1 -- 172.21.15.31:0/2826727963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55655d18f280 con 0x55655d234d40 2024-09-16T00:46:24.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.326+0000 7f163de2e740 1 -- 172.21.15.31:0/2826727963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55655d136b10 con 0x55655d229b60 2024-09-16T00:46:24.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.326+0000 7f163de2e740 1 -- 172.21.15.31:0/2826727963 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55655d18b240 con 0x55655d22a480 2024-09-16T00:46:24.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.328+0000 7f1639ff2640 1 -- 172.21.15.31:0/2826727963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 msgr2=0x55655d234800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:46:24.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.328+0000 7f1639ff2640 1 --2- 172.21.15.31:0/2826727963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 0x55655d234800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.328+0000 7f1639ff2640 1 -- 172.21.15.31:0/2826727963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 msgr2=0x55655d237130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:46:24.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.328+0000 7f1639ff2640 1 --2- 172.21.15.31:0/2826727963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 0x55655d237130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.328+0000 7f1639ff2640 1 -- 172.21.15.31:0/2826727963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55655d1431f0 con 0x55655d229b60 2024-09-16T00:46:24.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.330+0000 7f1639ff2640 1 --2- 172.21.15.31:0/2826727963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d229f40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f162c02ae20 tx=0x7f162c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ef173245fb84da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:24.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.331+0000 7f162a7fc640 1 -- 172.21.15.31:0/1609056605 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1624018a40 con 0x7f1634109830 2024-09-16T00:46:24.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.332+0000 7f162a7fc640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f16200a3450 0x7f16200a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.332+0000 7f16397f1640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f16200a3450 0x7f16200a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.333+0000 7f16397f1640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f16200a3450 0x7f16200a5910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55655d226f50 tx=0x7f163000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.333+0000 7f162a7fc640 1 -- 172.21.15.31:0/1609056605 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f16240cff80 con 0x7f1634109830 2024-09-16T00:46:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.337+0000 7f161f7fe640 1 -- 172.21.15.31:0/2826727963 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c031640 con 0x55655d229b60 2024-09-16T00:46:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.337+0000 7f161f7fe640 1 -- 172.21.15.31:0/2826727963 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f162c036070 con 0x55655d229b60 2024-09-16T00:46:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.337+0000 7f161f7fe640 1 -- 172.21.15.31:0/2826727963 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c030e10 con 0x55655d229b60 2024-09-16T00:46:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.339+0000 7f163de2e740 1 -- 172.21.15.31:0/2826727963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 msgr2=0x55655d229f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.339+0000 7f163de2e740 1 --2- 172.21.15.31:0/2826727963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d229f40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f162c02ae20 tx=0x7f162c009a20 comp rx=0 tx=0).stop 2024-09-16T00:46:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.341+0000 7f163de2e740 1 -- 172.21.15.31:0/2826727963 shutdown_connections 2024-09-16T00:46:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.341+0000 7f163de2e740 1 --2- 172.21.15.31:0/2826727963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 0x55655d237130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.341+0000 7f163de2e740 1 --2- 172.21.15.31:0/2826727963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 0x55655d234800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.341+0000 7f163de2e740 1 --2- 172.21.15.31:0/2826727963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d229f40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.341+0000 7f163de2e740 1 -- 172.21.15.31:0/2826727963 >> 172.21.15.31:0/2826727963 conn(0x55655d2260f0 msgr2=0x55655d2264d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.343+0000 7f163de2e740 1 -- 172.21.15.31:0/2826727963 shutdown_connections 2024-09-16T00:46:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.347+0000 7f163de2e740 1 -- 172.21.15.31:0/2826727963 wait complete. 2024-09-16T00:46:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.347+0000 7f163de2e740 1 -- start start 2024-09-16T00:46:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.347+0000 7f163de2e740 1 Processor -- start 2024-09-16T00:46:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.348+0000 7f163de2e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d231c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.348+0000 7f163de2e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 0x55655d2321d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.348+0000 7f163de2e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 0x55655d232710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.348+0000 7f1639ff2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 0x55655d232710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f1639ff2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 0x55655d232710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57312/0 (socket says 172.21.15.31:57312) 2024-09-16T00:46:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f1638ff0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 0x55655d2321d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f1639ff2640 1 -- 172.21.15.31:0/2042445313 learned_addr learned my addr 172.21.15.31:0/2042445313 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55655d237910 con 0x55655d234d40 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55655d149f10 con 0x55655d229b60 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55655d237a90 con 0x55655d22a480 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f1639ff2640 1 -- 172.21.15.31:0/2042445313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 msgr2=0x55655d2321d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f1639ff2640 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 0x55655d2321d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f1639ff2640 1 -- 172.21.15.31:0/2042445313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 msgr2=0x55655d231c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f1639ff2640 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d231c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.349+0000 7f1639ff2640 1 -- 172.21.15.31:0/2042445313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55655cf006a0 con 0x55655d234d40 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.350+0000 7f1639ff2640 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 0x55655d232710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f162c005e30 tx=0x7f162c0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85fdd9dda21ba94 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.350+0000 7f1638ff0640 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 0x55655d2321d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.350+0000 7f163de2e740 10 client.0 _ll_register_callbacks cb 0x55655d112a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.350+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f162c036070 con 0x55655d234d40 2024-09-16T00:46:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.350+0000 7f161cff9640 10 client.15975 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:46:24.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.350+0000 7f161cff9640 1 -- 172.21.15.31:0/2042445313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c030a70 con 0x55655d234d40 2024-09-16T00:46:24.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.350+0000 7f161cff9640 1 -- 172.21.15.31:0/2042445313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f162c030c10 con 0x55655d234d40 2024-09-16T00:46:24.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.353+0000 7f161cff9640 1 -- 172.21.15.31:0/2042445313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f162c030e10 con 0x55655d234d40 2024-09-16T00:46:24.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.354+0000 7f163de2e740 10 client.15975 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:24.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.354+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({fsmap=20}) -- 0x55655d254090 con 0x55655d234d40 2024-09-16T00:46:24.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.357+0000 7f161cff9640 1 -- 172.21.15.31:0/2042445313 <== mon.0 v2:172.21.15.31:3300/0 4 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f162c04d070 con 0x55655d234d40 2024-09-16T00:46:24.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.357+0000 7f163de2e740 10 client.15975 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:24.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.357+0000 7f163de2e740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:46:24.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.357+0000 7f163de2e740 20 client.15975 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:24.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.357+0000 7f163de2e740 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55655d255960 0x55655d257e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.357+0000 7f163de2e740 4 client.15975 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:46:24.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.357+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f162c036070 con 0x55655d255960 2024-09-16T00:46:24.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.361+0000 7f16397f1640 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55655d255960 0x55655d257e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.362+0000 7f16397f1640 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55655d255960 0x55655d257e20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d31bd66e44ddaf7 server_cookie=c32e38411228de7c in_seq=0 out_seq=0 2024-09-16T00:46:24.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.363+0000 7f161cff9640 10 client.15975 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:24.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.369+0000 7f161cff9640 1 -- 172.21.15.31:0/2042445313 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f162c036070 con 0x55655d255960 2024-09-16T00:46:24.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.369+0000 7f161cff9640 10 client.15975 handle_command_reply: tid=0 2024-09-16T00:46:24.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.369+0000 7f163de2e740 1 client.15975 shutdown 2024-09-16T00:46:24.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.379+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 msgr2=0x55655d232710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:24.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.379+0000 7f163de2e740 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 0x55655d232710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f162c005e30 tx=0x7f162c0305d0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.387+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 shutdown_connections 2024-09-16T00:46:24.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.387+0000 7f163de2e740 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d234d40 0x55655d232710 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.387+0000 7f163de2e740 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55655d255960 0x55655d257e20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.387+0000 7f163de2e740 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d22a480 0x55655d2321d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.387+0000 7f163de2e740 1 --2- 172.21.15.31:0/2042445313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d231c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.387+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 >> 172.21.15.31:0/2042445313 conn(0x55655d2260f0 msgr2=0x55655d225270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:24.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.387+0000 7f161cff9640 0 client.15975 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:24.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.391+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 shutdown_connections 2024-09-16T00:46:24.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.391+0000 7f163de2e740 1 -- 172.21.15.31:0/2042445313 wait complete. 2024-09-16T00:46:24.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.391+0000 7f163de2e740 20 client.15975 trim_cache size 0 max 16384 2024-09-16T00:46:24.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.436+0000 7f163de2e740 1 Processor -- start 2024-09-16T00:46:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.438+0000 7f163de2e740 1 -- start start 2024-09-16T00:46:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f163de2e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d229b60 0x55655d225a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f163de2e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d232560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f1638ff0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d232560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f16397f1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d229b60 0x55655d225a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f16397f1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d229b60 0x55655d225a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:46546/0 (socket says 172.21.15.31:46546) 2024-09-16T00:46:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f1638ff0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d232560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57328/0 (socket says 172.21.15.31:57328) 2024-09-16T00:46:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f163de2e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d234d40 0x55655d25d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f16397f1640 1 -- 172.21.15.31:0/1103166540 learned_addr learned my addr 172.21.15.31:0/1103166540 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f163de2e740 1 -- 172.21.15.31:0/1103166540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f161800daf0 con 0x55655d22a480 2024-09-16T00:46:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f163de2e740 1 -- 172.21.15.31:0/1103166540 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1618010370 con 0x55655d234d40 2024-09-16T00:46:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.439+0000 7f163de2e740 1 -- 172.21.15.31:0/1103166540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f161800e160 con 0x55655d229b60 2024-09-16T00:46:24.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.441+0000 7f1639ff2640 1 --2- 172.21.15.31:0/1103166540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d234d40 0x55655d25d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.442+0000 7f16397f1640 1 -- 172.21.15.31:0/1103166540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d234d40 msgr2=0x55655d25d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.442+0000 7f16397f1640 1 --2- 172.21.15.31:0/1103166540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d234d40 0x55655d25d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.442+0000 7f16397f1640 1 -- 172.21.15.31:0/1103166540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 msgr2=0x55655d232560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.442+0000 7f16397f1640 1 --2- 172.21.15.31:0/1103166540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d232560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.442+0000 7f16397f1640 1 -- 172.21.15.31:0/1103166540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55655d234560 con 0x55655d229b60 2024-09-16T00:46:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.443+0000 7f16397f1640 1 --2- 172.21.15.31:0/1103166540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d229b60 0x55655d225a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f163000fe20 tx=0x7f1630012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb33ccacfeb97467 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.447+0000 7f161effd640 1 -- 172.21.15.31:0/1103166540 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163002a030 con 0x55655d229b60 2024-09-16T00:46:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.447+0000 7f161effd640 1 -- 172.21.15.31:0/1103166540 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1630025330 con 0x55655d229b60 2024-09-16T00:46:24.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.447+0000 7f161effd640 1 -- 172.21.15.31:0/1103166540 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630016070 con 0x55655d229b60 2024-09-16T00:46:24.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.448+0000 7f163de2e740 1 -- 172.21.15.31:0/1103166540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d229b60 msgr2=0x55655d225a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:24.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.448+0000 7f163de2e740 1 --2- 172.21.15.31:0/1103166540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d229b60 0x55655d225a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f163000fe20 tx=0x7f1630012000 comp rx=0 tx=0).stop 2024-09-16T00:46:24.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.452+0000 7f163de2e740 1 -- 172.21.15.31:0/1103166540 shutdown_connections 2024-09-16T00:46:24.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.452+0000 7f163de2e740 1 --2- 172.21.15.31:0/1103166540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d234d40 0x55655d25d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.452+0000 7f163de2e740 1 --2- 172.21.15.31:0/1103166540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d232560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.452+0000 7f163de2e740 1 --2- 172.21.15.31:0/1103166540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d229b60 0x55655d225a60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.452+0000 7f163de2e740 1 -- 172.21.15.31:0/1103166540 >> 172.21.15.31:0/1103166540 conn(0x55655d2260f0 msgr2=0x55655d233c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:24.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.453+0000 7f163de2e740 1 -- 172.21.15.31:0/1103166540 shutdown_connections 2024-09-16T00:46:24.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.453+0000 7f163de2e740 1 -- 172.21.15.31:0/1103166540 wait complete. 2024-09-16T00:46:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.454+0000 7f163de2e740 1 -- start start 2024-09-16T00:46:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.459+0000 7f163de2e740 1 Processor -- start 2024-09-16T00:46:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.462+0000 7f163de2e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d23cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.464+0000 7f163de2e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d23d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.464+0000 7f163de2e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d234d40 0x55655d239d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.464+0000 7f16397f1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d23cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.464+0000 7f16397f1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d23cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:43564/0 (socket says 172.21.15.31:43564) 2024-09-16T00:46:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f1638ff0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d23d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f16397f1640 1 -- 172.21.15.31:0/1341455833 learned_addr learned my addr 172.21.15.31:0/1341455833 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f161800e830 con 0x55655d22a480 2024-09-16T00:46:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f161800fca0 con 0x55655d229b60 2024-09-16T00:46:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f161800ef00 con 0x55655d234d40 2024-09-16T00:46:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f163de2e740 10 client.0 _ll_register_callbacks cb 0x7f1624005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f1639ff2640 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d234d40 0x55655d239d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f1638ff0640 1 -- 172.21.15.31:0/1341455833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d234d40 msgr2=0x55655d239d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f1638ff0640 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d234d40 0x55655d239d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f1638ff0640 1 -- 172.21.15.31:0/1341455833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 msgr2=0x55655d23cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:24.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f1638ff0640 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d23cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:24.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.465+0000 7f1638ff0640 1 -- 172.21.15.31:0/1341455833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55655d23a280 con 0x55655d22a480 2024-09-16T00:46:24.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.466+0000 7f1638ff0640 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d23d2b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55655d227470 tx=0x7f162409d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8dac795e39dfe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:24.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.466+0000 7f161cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:46:24.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.467+0000 7f161cff9640 1 -- 172.21.15.31:0/1341455833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240b5060 con 0x55655d22a480 2024-09-16T00:46:24.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.467+0000 7f161cff9640 1 -- 172.21.15.31:0/1341455833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f16240a10d0 con 0x55655d22a480 2024-09-16T00:46:24.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.467+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1630025330 con 0x55655d22a480 2024-09-16T00:46:24.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.468+0000 7f161cff9640 1 -- 172.21.15.31:0/1341455833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f16240a5e80 con 0x55655d22a480 2024-09-16T00:46:24.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.468+0000 7f163de2e740 10 client.15981 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:24.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.468+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55655d2c0f50 con 0x55655d22a480 2024-09-16T00:46:24.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.471+0000 7f161cff9640 1 -- 172.21.15.31:0/1341455833 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16240a12d0 con 0x55655d22a480 2024-09-16T00:46:24.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.471+0000 7f161cff9640 1 -- 172.21.15.31:0/1341455833 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f16240cdb30 con 0x55655d22a480 2024-09-16T00:46:24.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.472+0000 7f163de2e740 10 client.15981 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.472+0000 7f163de2e740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:46:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.472+0000 7f163de2e740 20 client.15981 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.472+0000 7f163de2e740 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55655d2c1620 0x55655d2c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.472+0000 7f163de2e740 4 client.15981 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:46:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.472+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f1630025330 con 0x55655d2c1620 2024-09-16T00:46:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.472+0000 7f16397f1640 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55655d2c1620 0x55655d2c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.473+0000 7f16397f1640 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55655d2c1620 0x55655d2c3a40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c24c7faf500148 server_cookie=ed14faeec47c59c3 in_seq=0 out_seq=0 2024-09-16T00:46:24.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:24.473+0000 7f161cff9640 10 client.15981 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:25.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.042+0000 7f161cff9640 1 -- 172.21.15.31:0/1341455833 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f1630025330 con 0x55655d2c1620 2024-09-16T00:46:25.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.042+0000 7f161cff9640 10 client.15981 handle_command_reply: tid=0 2024-09-16T00:46:25.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.043+0000 7f163de2e740 1 client.15981 shutdown 2024-09-16T00:46:25.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:24 smithi031 ceph-mon[30956]: pgmap v514: 129 pgs: 129 active+clean; 589 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 424 op/s 2024-09-16T00:46:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.045+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 msgr2=0x55655d23d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.046+0000 7f163de2e740 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d23d2b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55655d227470 tx=0x7f162409d910 comp rx=0 tx=0).stop 2024-09-16T00:46:25.055 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:46:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:46:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "26b8bd8b-0318-46a0-8211-5666bd5eda82", 2024-09-16T00:46:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:46:25.056 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:46:25.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.046+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 shutdown_connections 2024-09-16T00:46:25.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.046+0000 7f163de2e740 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55655d2c1620 0x55655d2c3a40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.046+0000 7f163de2e740 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655d234d40 0x55655d239d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.046+0000 7f161cff9640 0 client.15981 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.046+0000 7f163de2e740 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55655d22a480 0x55655d23d2b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.046+0000 7f163de2e740 1 --2- 172.21.15.31:0/1341455833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55655d229b60 0x55655d23cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.046+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 >> 172.21.15.31:0/1341455833 conn(0x55655d2260f0 msgr2=0x55655d22b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.047+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 shutdown_connections 2024-09-16T00:46:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.047+0000 7f163de2e740 1 -- 172.21.15.31:0/1341455833 wait complete. 2024-09-16T00:46:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.047+0000 7f163de2e740 20 client.15981 trim_cache size 0 max 16384 2024-09-16T00:46:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 -- 172.21.15.31:0/1609056605 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f16200a3450 msgr2=0x7f16200a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f16200a3450 0x7f16200a5910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55655d226f50 tx=0x7f163000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 -- 172.21.15.31:0/1609056605 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1634109830 msgr2=0x7f16341a76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1634109830 0x7f16341a76a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f162400e9f0 tx=0x7f162400edf0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 -- 172.21.15.31:0/1609056605 shutdown_connections 2024-09-16T00:46:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f16200a3450 0x7f16200a5910 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f163411b140 0x7f16341aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1634109830 0x7f16341a76a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 --2- 172.21.15.31:0/1609056605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1634109450 0x7f16341a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.048+0000 7f163ba55640 1 -- 172.21.15.31:0/1609056605 >> 172.21.15.31:0/1609056605 conn(0x7f163406e140 msgr2=0x7f163411bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:25.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.049+0000 7f163ba55640 1 -- 172.21.15.31:0/1609056605 shutdown_connections 2024-09-16T00:46:25.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.049+0000 7f163ba55640 1 -- 172.21.15.31:0/1609056605 wait complete. 2024-09-16T00:46:25.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:24 smithi081 ceph-mon[35539]: pgmap v514: 129 pgs: 129 active+clean; 589 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 424 op/s 2024-09-16T00:46:25.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:24 smithi102 ceph-mon[35364]: pgmap v514: 129 pgs: 129 active+clean; 589 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 424 op/s 2024-09-16T00:46:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.367+0000 7f2a7a0f8640 1 -- 172.21.15.31:0/1478815671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a74106a90 msgr2=0x7f2a74106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.367+0000 7f2a7a0f8640 1 --2- 172.21.15.31:0/1478815671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a74106a90 0x7f2a74106e70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009900 tx=0x7f2a6402a420 comp rx=0 tx=0).stop 2024-09-16T00:46:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.368+0000 7f2a7a0f8640 1 -- 172.21.15.31:0/1478815671 shutdown_connections 2024-09-16T00:46:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.368+0000 7f2a7a0f8640 1 --2- 172.21.15.31:0/1478815671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a74100860 0x7f2a741045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.368+0000 7f2a7a0f8640 1 --2- 172.21.15.31:0/1478815671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a74107440 0x7f2a74100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.368+0000 7f2a7a0f8640 1 --2- 172.21.15.31:0/1478815671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a74106a90 0x7f2a74106e70 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009900 tx=0x7f2a6402a420 comp rx=0 tx=0).stop 2024-09-16T00:46:25.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.368+0000 7f2a7a0f8640 1 -- 172.21.15.31:0/1478815671 >> 172.21.15.31:0/1478815671 conn(0x7f2a74077230 msgr2=0x7f2a740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.373+0000 7f2a7a0f8640 1 -- 172.21.15.31:0/1478815671 shutdown_connections 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.374+0000 7f2a7a0f8640 1 -- 172.21.15.31:0/1478815671 wait complete. 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.375+0000 7f2a7a0f8640 1 Processor -- start 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.375+0000 7f2a7a0f8640 1 -- start start 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.375+0000 7f2a7a0f8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a74100860 0x7f2a7410cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.375+0000 7f2a7a0f8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a7410d020 0x7f2a7410b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.375+0000 7f2a7a0f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7410bdb0 0x7f2a7410c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.375+0000 7f2a7a0f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7406c1e0 con 0x7f2a7410bdb0 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.375+0000 7f2a7a0f8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7406c060 con 0x7f2a74100860 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.375+0000 7f2a7a0f8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7406c360 con 0x7f2a7410d020 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.376+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a7410d020 0x7f2a7410b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.376+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a7410d020 0x7f2a7410b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:46566/0 (socket says 172.21.15.31:46566) 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.377+0000 7f2a72ffd640 1 -- 172.21.15.31:0/1567985328 learned_addr learned my addr 172.21.15.31:0/1567985328 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.377+0000 7f2a737fe640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a74100860 0x7f2a7410cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.377+0000 7f2a72ffd640 1 -- 172.21.15.31:0/1567985328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a74100860 msgr2=0x7f2a7410cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.377+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a74100860 0x7f2a7410cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.377+0000 7f2a72ffd640 1 -- 172.21.15.31:0/1567985328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7410bdb0 msgr2=0x7f2a7410c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.377+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7410bdb0 0x7f2a7410c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.377+0000 7f2a72ffd640 1 -- 172.21.15.31:0/1567985328 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a74113b60 con 0x7f2a7410d020 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.382+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a7410d020 0x7f2a7410b870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800d770 tx=0x7f2a6800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.386+0000 7f2a70ff9640 1 -- 172.21.15.31:0/1567985328 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68017070 con 0x7f2a7410d020 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.386+0000 7f2a70ff9640 1 -- 172.21.15.31:0/1567985328 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a68009c30 con 0x7f2a7410d020 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.386+0000 7f2a70ff9640 1 -- 172.21.15.31:0/1567985328 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800f6b0 con 0x7f2a7410d020 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.387+0000 7f2a7a0f8640 1 -- 172.21.15.31:0/1567985328 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a74113e50 con 0x7f2a7410d020 2024-09-16T00:46:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.388+0000 7f2a7a0f8640 1 -- 172.21.15.31:0/1567985328 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a74114390 con 0x7f2a7410d020 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.399+0000 7f2a7c4d1740 1 Processor -- start 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.399+0000 7f2a7c4d1740 1 -- start start 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.399+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c0180 0x55bc201be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.399+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201bee90 0x55bc201bf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.399+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 0x55bc201cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.399+0000 7f2a7c4d1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bc20123160 con 0x55bc201c97f0 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.399+0000 7f2a7c4d1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bc200ca720 con 0x55bc201bee90 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.399+0000 7f2a7c4d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bc2011f120 con 0x55bc201c0180 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.402+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 0x55bc201cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.402+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 0x55bc201cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57352/0 (socket says 172.21.15.31:57352) 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.402+0000 7f2a72ffd640 1 -- 172.21.15.31:0/3759525631 learned_addr learned my addr 172.21.15.31:0/3759525631 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.404+0000 7f2a70ff9640 1 -- 172.21.15.31:0/1567985328 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2a6801a070 con 0x7f2a7410d020 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.405+0000 7f2a73fff640 1 --2- 172.21.15.31:0/3759525631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c0180 0x55bc201be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.410+0000 7f2a737fe640 1 --2- 172.21.15.31:0/3759525631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201bee90 0x55bc201bf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.412+0000 7f2a70ff9640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2a5c0a33d0 0x7f2a5c0a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.413+0000 7f2a70ff9640 1 -- 172.21.15.31:0/1567985328 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f2a68009de0 con 0x7f2a7410d020 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.414+0000 7f2a73fff640 1 -- 172.21.15.31:0/3759525631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201bee90 msgr2=0x55bc201bf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.414+0000 7f2a73fff640 1 --2- 172.21.15.31:0/3759525631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201bee90 0x55bc201bf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.414+0000 7f2a73fff640 1 -- 172.21.15.31:0/3759525631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 msgr2=0x55bc201cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.414+0000 7f2a73fff640 1 --2- 172.21.15.31:0/3759525631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 0x55bc201cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.414+0000 7f2a73fff640 1 -- 172.21.15.31:0/3759525631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc1fe94ba0 con 0x55bc201c0180 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.414+0000 7f2a73fff640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2a5c0a33d0 0x7f2a5c0a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.414+0000 7f2a73fff640 1 --2- 172.21.15.31:0/3759525631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c0180 0x55bc201be950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005fd0 tx=0x7f2a6c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5a61d74031aada4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.415+0000 7f2a7a0f8640 1 -- 172.21.15.31:0/3759525631 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c012590 con 0x55bc201c0180 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.415+0000 7f2a7a0f8640 1 -- 172.21.15.31:0/3759525631 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6c012c10 con 0x55bc201c0180 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.417+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3759525631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c0180 msgr2=0x55bc201be950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.417+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3759525631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c0180 0x55bc201be950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005fd0 tx=0x7f2a6c010740 comp rx=0 tx=0).stop 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.417+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3759525631 shutdown_connections 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.417+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3759525631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 0x55bc201cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.417+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3759525631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201bee90 0x55bc201bf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.417+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3759525631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c0180 0x55bc201be950 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.417+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3759525631 >> 172.21.15.31:0/3759525631 conn(0x55bc201bb0e0 msgr2=0x55bc201bb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.418+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3759525631 shutdown_connections 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.418+0000 7f2a73fff640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2a5c0a33d0 0x7f2a5c0a5890 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005fd0 tx=0x7f2a6c010740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.423+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3759525631 wait complete. 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.423+0000 7f2a7c4d1740 1 -- start start 2024-09-16T00:46:25.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.424+0000 7f2a7c4d1740 1 Processor -- start 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.425+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201bee90 0x55bc201c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.425+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 0x55bc201c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.425+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c97f0 0x55bc201c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.425+0000 7f2a7c4d1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bc201cc6b0 con 0x55bc201bee90 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.425+0000 7f2a7c4d1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bc200ab900 con 0x55bc201c0180 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.425+0000 7f2a7c4d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bc201cc830 con 0x55bc201c97f0 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.425+0000 7f2a7c4d1740 10 client.0 _ll_register_callbacks cb 0x55bc200a9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.429+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201bee90 0x55bc201c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.429+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201bee90 0x55bc201c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57366/0 (socket says 172.21.15.31:57366) 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.429+0000 7f2a737fe640 1 -- 172.21.15.31:0/812094073 learned_addr learned my addr 172.21.15.31:0/812094073 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.429+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 0x55bc201c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.430+0000 7f2a73fff640 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c97f0 0x55bc201c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.431+0000 7f2a73fff640 1 -- 172.21.15.31:0/812094073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 msgr2=0x55bc201c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.431+0000 7f2a73fff640 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 0x55bc201c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.431+0000 7f2a73fff640 1 -- 172.21.15.31:0/812094073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201bee90 msgr2=0x55bc201c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.431+0000 7f2a73fff640 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201bee90 0x55bc201c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.431+0000 7f2a73fff640 1 -- 172.21.15.31:0/812094073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc200ebc50 con 0x55bc201c97f0 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.433+0000 7f2a73fff640 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c97f0 0x55bc201c70b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c01afd0 tx=0x7f2a6c009a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74d2e34daf6f179 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.434+0000 7f2a52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.438+0000 7f2a52ffd640 1 -- 172.21.15.31:0/812094073 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c028070 con 0x55bc201c97f0 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.438+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/812094073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6c012c10 con 0x55bc201c97f0 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.438+0000 7f2a52ffd640 1 -- 172.21.15.31:0/812094073 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f2a6c011850 con 0x55bc201c97f0 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.440+0000 7f2a52ffd640 1 -- 172.21.15.31:0/812094073 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2a6c03c050 con 0x55bc201c97f0 2024-09-16T00:46:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.440+0000 7f2a7c4d1740 10 client.25591 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.440+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/812094073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bc201d8340 con 0x55bc201c97f0 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.445+0000 7f2a52ffd640 1 -- 172.21.15.31:0/812094073 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6c012c10 con 0x55bc201c97f0 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.445+0000 7f2a52ffd640 1 -- 172.21.15.31:0/812094073 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f2a6c02f470 con 0x55bc201c97f0 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.446+0000 7f2a7c4d1740 10 client.25591 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.446+0000 7f2a7c4d1740 10 client.25591 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.446+0000 7f2a7c4d1740 20 client.25591 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.446+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bc201d9b20 0x55bc201dbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.446+0000 7f2a7c4d1740 4 client.25591 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.446+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/812094073 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc20130890 con 0x55bc201d9b20 2024-09-16T00:46:25.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.447+0000 7f2a737fe640 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bc201d9b20 0x55bc201dbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.455+0000 7f2a737fe640 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bc201d9b20 0x55bc201dbf90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da569bd2eec97471 server_cookie=66f4992ded188326 in_seq=0 out_seq=0 2024-09-16T00:46:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.455+0000 7f2a52ffd640 10 client.25591 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:25.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.460+0000 7f2a52ffd640 1 -- 172.21.15.31:0/812094073 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc20130890 con 0x55bc201d9b20 2024-09-16T00:46:25.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.460+0000 7f2a52ffd640 10 client.25591 handle_command_reply: tid=0 2024-09-16T00:46:25.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.460+0000 7f2a7c4d1740 1 client.25591 shutdown 2024-09-16T00:46:25.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.466+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/812094073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c97f0 msgr2=0x55bc201c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:25.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.466+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c97f0 0x55bc201c70b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c01afd0 tx=0x7f2a6c009a00 comp rx=0 tx=0).stop 2024-09-16T00:46:25.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.467+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/812094073 shutdown_connections 2024-09-16T00:46:25.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.467+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bc201d9b20 0x55bc201dbf90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.467+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201c97f0 0x55bc201c70b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.467+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 0x55bc201c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.467+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/812094073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201bee90 0x55bc201c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.467+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/812094073 >> 172.21.15.31:0/812094073 conn(0x55bc201bb0e0 msgr2=0x55bc201c05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:25.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.467+0000 7f2a52ffd640 0 client.25591 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:25.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.471+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/812094073 shutdown_connections 2024-09-16T00:46:25.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.473+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/812094073 wait complete. 2024-09-16T00:46:25.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.473+0000 7f2a7c4d1740 20 client.25591 trim_cache size 0 max 16384 2024-09-16T00:46:25.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.543+0000 7f2a7c4d1740 1 Processor -- start 2024-09-16T00:46:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.544+0000 7f2a7c4d1740 1 -- start start 2024-09-16T00:46:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.544+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.544+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 0x55bc201e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.545+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 0x55bc201e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.545+0000 7f2a7c4d1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741c4b00 con 0x55bc201c97f0 2024-09-16T00:46:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.545+0000 7f2a7c4d1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741c7380 con 0x55bc201c0180 2024-09-16T00:46:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.545+0000 7f2a7c4d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741c5170 con 0x55bc201bee90 2024-09-16T00:46:25.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.545+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.545+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201c0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:46604/0 (socket says 172.21.15.31:46604) 2024-09-16T00:46:25.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.545+0000 7f2a737fe640 1 -- 172.21.15.31:0/1844629724 learned_addr learned my addr 172.21.15.31:0/1844629724 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:25.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.546+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/1844629724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 0x55bc201e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.547+0000 7f2a72ffd640 1 -- 172.21.15.31:0/1844629724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 msgr2=0x55bc201c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.547+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/1844629724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.547+0000 7f2a72ffd640 1 -- 172.21.15.31:0/1844629724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 msgr2=0x55bc201e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:25.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.547+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/1844629724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 0x55bc201e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.547+0000 7f2a72ffd640 1 -- 172.21.15.31:0/1844629724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc201c8fa0 con 0x55bc201c0180 2024-09-16T00:46:25.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.548+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/1844629724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 0x55bc201e1830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800d740 tx=0x7f2a680d2450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7d81edebc47d33b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:25.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.551+0000 7f2a597fa640 1 -- 172.21.15.31:0/1844629724 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680a10a0 con 0x55bc201c0180 2024-09-16T00:46:25.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.551+0000 7f2a597fa640 1 -- 172.21.15.31:0/1844629724 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a680a5450 con 0x55bc201c0180 2024-09-16T00:46:25.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.551+0000 7f2a597fa640 1 -- 172.21.15.31:0/1844629724 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6809d680 con 0x55bc201c0180 2024-09-16T00:46:25.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.554+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/1844629724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 msgr2=0x55bc201e1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:25.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.554+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/1844629724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 0x55bc201e1830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800d740 tx=0x7f2a680d2450 comp rx=0 tx=0).stop 2024-09-16T00:46:25.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.557+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/1844629724 shutdown_connections 2024-09-16T00:46:25.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.557+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/1844629724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c97f0 0x55bc201e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.557+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/1844629724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c0180 0x55bc201e1830 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.557+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/1844629724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.557+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/1844629724 >> 172.21.15.31:0/1844629724 conn(0x55bc201bb0e0 msgr2=0x55bc201c05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.563+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/1844629724 shutdown_connections 2024-09-16T00:46:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.563+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/1844629724 wait complete. 2024-09-16T00:46:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.564+0000 7f2a7c4d1740 1 -- start start 2024-09-16T00:46:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.565+0000 7f2a7c4d1740 1 Processor -- start 2024-09-16T00:46:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.565+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c0180 0x55bc201d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a7c4d1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c97f0 0x55bc201c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201d1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:46620/0 (socket says 172.21.15.31:46620) 2024-09-16T00:46:25.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a737fe640 1 -- 172.21.15.31:0/3434847233 learned_addr learned my addr 172.21.15.31:0/3434847233 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741c5840 con 0x55bc201c0180 2024-09-16T00:46:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a73fff640 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c97f0 0x55bc201c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741c6cb0 con 0x55bc201c97f0 2024-09-16T00:46:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741c5f10 con 0x55bc201bee90 2024-09-16T00:46:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.566+0000 7f2a7c4d1740 10 client.0 _ll_register_callbacks cb 0x7f2a6c01d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:25.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.568+0000 7f2a737fe640 1 -- 172.21.15.31:0/3434847233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c97f0 msgr2=0x55bc201c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:25.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.568+0000 7f2a737fe640 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c97f0 0x55bc201c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.568+0000 7f2a737fe640 1 -- 172.21.15.31:0/3434847233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c0180 msgr2=0x55bc201d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:25.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.568+0000 7f2a737fe640 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c0180 0x55bc201d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:25.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.568+0000 7f2a737fe640 1 -- 172.21.15.31:0/3434847233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc201c3840 con 0x55bc201bee90 2024-09-16T00:46:25.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.577+0000 7f2a737fe640 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201d1e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2a6402aa10 tx=0x7f2a64004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60407b13519ff777 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:25.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.581+0000 7f2a52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:46:25.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.581+0000 7f2a52ffd640 1 -- 172.21.15.31:0/3434847233 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64035040 con 0x55bc201bee90 2024-09-16T00:46:25.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.581+0000 7f2a52ffd640 1 -- 172.21.15.31:0/3434847233 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f2a64031400 con 0x55bc201bee90 2024-09-16T00:46:25.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.584+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a680a5450 con 0x55bc201bee90 2024-09-16T00:46:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.588+0000 7f2a52ffd640 1 -- 172.21.15.31:0/3434847233 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2a64031600 con 0x55bc201bee90 2024-09-16T00:46:25.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.590+0000 7f2a7c4d1740 10 client.25597 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:25.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.590+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bc20244400 con 0x55bc201bee90 2024-09-16T00:46:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.597+0000 7f2a52ffd640 1 -- 172.21.15.31:0/3434847233 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a64005520 con 0x55bc201bee90 2024-09-16T00:46:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.597+0000 7f2a52ffd640 1 -- 172.21.15.31:0/3434847233 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f2a64004c50 con 0x55bc201bee90 2024-09-16T00:46:25.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.599+0000 7f2a7c4d1740 10 client.25597 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:25.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.599+0000 7f2a7c4d1740 10 client.25597 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:46:25.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.599+0000 7f2a7c4d1740 20 client.25597 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:25.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.599+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bc20244b20 0x55bc20246f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:25.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.599+0000 7f2a7c4d1740 4 client.25597 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-16T00:46:25.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.599+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x7f2a680a5450 con 0x55bc20244b20 2024-09-16T00:46:25.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.609+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bc20244b20 0x55bc20246f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:25.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.618+0000 7f2a52ffd640 10 client.25597 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:25.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:25.618+0000 7f2a72ffd640 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bc20244b20 0x55bc20246f40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a2198d01eeaf55b server_cookie=4cd9e4bdb70a6cb4 in_seq=0 out_seq=0 2024-09-16T00:46:25.871 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:25 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:46:25.871 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:25 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:46:25.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:25 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:46:26.189 INFO:tasks.workunit.client.0.smithi031.stdout: 113 1924048 601 1251779 431 1136384 1734 2024-09-16T00:46:26.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:46:26.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:25 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:46:26.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:25 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:46:26.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:25 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:46:26.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:25 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:46:26.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:25 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:46:26.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:25 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:46:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:26 smithi081 ceph-mon[35539]: pgmap v515: 129 pgs: 129 active+clean; 580 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 350 op/s 2024-09-16T00:46:27.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:26 smithi102 ceph-mon[35364]: pgmap v515: 129 pgs: 129 active+clean; 580 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 350 op/s 2024-09-16T00:46:27.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:26 smithi031 ceph-mon[30956]: pgmap v515: 129 pgs: 129 active+clean; 580 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 350 op/s 2024-09-16T00:46:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:29.140 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:28 smithi031 ceph-mon[30956]: pgmap v516: 129 pgs: 129 active+clean; 581 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-09-16T00:46:29.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:28 smithi081 ceph-mon[35539]: pgmap v516: 129 pgs: 129 active+clean; 581 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-09-16T00:46:29.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:28 smithi102 ceph-mon[35364]: pgmap v516: 129 pgs: 129 active+clean; 581 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-09-16T00:46:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.994+0000 7fdb6d65e640 1 -- 172.21.15.31:0/4176260638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68106f10 msgr2=0x7fdb6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.994+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/4176260638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68106f10 0x7fdb6810ea90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009ca0 tx=0x7fdb6002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.994+0000 7fdb6d65e640 1 -- 172.21.15.31:0/4176260638 shutdown_connections 2024-09-16T00:46:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.994+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/4176260638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68106f10 0x7fdb6810ea90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.994+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/4176260638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb68102790 0x7fdb681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.994+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/4176260638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb68101de0 0x7fdb681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.994+0000 7fdb6d65e640 1 -- 172.21.15.31:0/4176260638 >> 172.21.15.31:0/4176260638 conn(0x7fdb680fd880 msgr2=0x7fdb680ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.994+0000 7fdb6d65e640 1 -- 172.21.15.31:0/4176260638 shutdown_connections 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.994+0000 7fdb6d65e640 1 -- 172.21.15.31:0/4176260638 wait complete. 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb6d65e640 1 Processor -- start 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb6d65e640 1 -- start start 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb6d65e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68101de0 0x7fdb681983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb6d65e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb68102790 0x7fdb681988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb6d65e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb68106f10 0x7fdb6819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb6d65e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68111f70 con 0x7fdb68101de0 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb6d65e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68111df0 con 0x7fdb68106f10 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb6d65e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681120f0 con 0x7fdb68102790 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb66ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68101de0 0x7fdb681983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb66ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68101de0 0x7fdb681983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43696/0 (socket says 172.21.15.31:43696) 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb66ffd640 1 -- 172.21.15.31:0/3462484947 learned_addr learned my addr 172.21.15.31:0/3462484947 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb677fe640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb68106f10 0x7fdb6819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb5ffff640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb68102790 0x7fdb681988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb66ffd640 1 -- 172.21.15.31:0/3462484947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb68102790 msgr2=0x7fdb681988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb66ffd640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb68102790 0x7fdb681988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb66ffd640 1 -- 172.21.15.31:0/3462484947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb68106f10 msgr2=0x7fdb6819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb66ffd640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb68106f10 0x7fdb6819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.995+0000 7fdb66ffd640 1 -- 172.21.15.31:0/3462484947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6819d360 con 0x7fdb68101de0 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.996+0000 7fdb66ffd640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68101de0 0x7fdb681983b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800e9f0 tx=0x7fdb5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.996+0000 7fdb64ff9640 1 -- 172.21.15.31:0/3462484947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800cd80 con 0x7fdb68101de0 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.996+0000 7fdb64ff9640 1 -- 172.21.15.31:0/3462484947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb58010910 con 0x7fdb68101de0 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.996+0000 7fdb6d65e640 1 -- 172.21.15.31:0/3462484947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6819d650 con 0x7fdb68101de0 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.996+0000 7fdb64ff9640 1 -- 172.21.15.31:0/3462484947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58018990 con 0x7fdb68101de0 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.996+0000 7fdb6d65e640 1 -- 172.21.15.31:0/3462484947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb681a4f30 con 0x7fdb68101de0 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.998+0000 7fdb64ff9640 1 -- 172.21.15.31:0/3462484947 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fdb58010ac0 con 0x7fdb68101de0 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.998+0000 7fdb6fa37740 1 Processor -- start 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.998+0000 7fdb6fa37740 1 -- start start 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf320880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf320dc0 0x5590bf32b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf32b680 0x5590bf32da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5590bf2862e0 con 0x5590bf3204a0 2024-09-16T00:46:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5590bf22dce0 con 0x5590bf32b680 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5590bf2822a0 con 0x5590bf320dc0 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf320880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf320880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43712/0 (socket says 172.21.15.31:43712) 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb677fe640 1 -- 172.21.15.31:0/1879144 learned_addr learned my addr 172.21.15.31:0/1879144 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb5ffff640 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf32b680 0x5590bf32da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb66ffd640 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf320dc0 0x5590bf32b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb677fe640 1 -- 172.21.15.31:0/1879144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf320dc0 msgr2=0x5590bf32b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb677fe640 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf320dc0 0x5590bf32b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb677fe640 1 -- 172.21.15.31:0/1879144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf32b680 msgr2=0x5590bf32da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb677fe640 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf32b680 0x5590bf32da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb677fe640 1 -- 172.21.15.31:0/1879144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590bf212440 con 0x5590bf3204a0 2024-09-16T00:46:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb5ffff640 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf32b680 0x5590bf32da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:28.999+0000 7fdb66ffd640 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf320dc0 0x5590bf32b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.000+0000 7fdb64ff9640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb540a3240 0x7fdb540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.000+0000 7fdb66ffd640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb540a3240 0x7fdb540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.000+0000 7fdb677fe640 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf320880 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009ca0 tx=0x7fdb60005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fee68b09aa1d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.000+0000 7fdb64ff9640 1 -- 172.21.15.31:0/3462484947 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fdb580cffb0 con 0x7fdb68101de0 2024-09-16T00:46:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.000+0000 7fdb5d7fa640 1 -- 172.21.15.31:0/1879144 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60031db0 con 0x5590bf3204a0 2024-09-16T00:46:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.000+0000 7fdb5d7fa640 1 -- 172.21.15.31:0/1879144 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb60036040 con 0x5590bf3204a0 2024-09-16T00:46:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.000+0000 7fdb5d7fa640 1 -- 172.21.15.31:0/1879144 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb600042a0 con 0x5590bf3204a0 2024-09-16T00:46:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb6fa37740 1 -- 172.21.15.31:0/1879144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 msgr2=0x5590bf320880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb66ffd640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb540a3240 0x7fdb540a5700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdb580cda70 tx=0x7fdb580d3a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf320880 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009ca0 tx=0x7fdb60005f10 comp rx=0 tx=0).stop 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb6fa37740 1 -- 172.21.15.31:0/1879144 shutdown_connections 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf32b680 0x5590bf32da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf320dc0 0x5590bf32b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/1879144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf320880 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb6fa37740 1 -- 172.21.15.31:0/1879144 >> 172.21.15.31:0/1879144 conn(0x5590bf31ca30 msgr2=0x5590bf31ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb6fa37740 1 -- 172.21.15.31:0/1879144 shutdown_connections 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.001+0000 7fdb6fa37740 1 -- 172.21.15.31:0/1879144 wait complete. 2024-09-16T00:46:29.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb6fa37740 1 -- start start 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb6fa37740 1 Processor -- start 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf32af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf3281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf32b680 0x5590bf328730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5590bf32e0d0 con 0x5590bf3204a0 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf32af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5590bf241050 con 0x5590bf320dc0 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5590bf32e250 con 0x5590bf32b680 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf32af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43726/0 (socket says 172.21.15.31:43726) 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb677fe640 1 -- 172.21.15.31:0/3069922909 learned_addr learned my addr 172.21.15.31:0/3069922909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb6fa37740 10 client.0 _ll_register_callbacks cb 0x5590bf281160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb5ffff640 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf3281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb677fe640 1 -- 172.21.15.31:0/3069922909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf32b680 msgr2=0x5590bf328730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:46:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb677fe640 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf32b680 0x5590bf328730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb677fe640 1 -- 172.21.15.31:0/3069922909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 msgr2=0x5590bf3281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb677fe640 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf3281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.002+0000 7fdb677fe640 1 -- 172.21.15.31:0/3069922909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590beff73c0 con 0x5590bf3204a0 2024-09-16T00:46:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb677fe640 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf32af40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdb60005cf0 tx=0x7fdb600318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cf90dce721ae4d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:46:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/3069922909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60031450 con 0x5590bf3204a0 2024-09-16T00:46:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/3069922909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fdb60004040 con 0x5590bf3204a0 2024-09-16T00:46:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3069922909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb60036040 con 0x5590bf3204a0 2024-09-16T00:46:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/3069922909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdb6005c050 con 0x5590bf3204a0 2024-09-16T00:46:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb6fa37740 10 client.16002 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3069922909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5590bf32e0d0 con 0x5590bf3204a0 2024-09-16T00:46:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/3069922909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb60036040 con 0x5590bf3204a0 2024-09-16T00:46:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/3069922909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fdb60057030 con 0x5590bf3204a0 2024-09-16T00:46:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb6fa37740 10 client.16002 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb6fa37740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb6fa37740 20 client.16002 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bf33efe0 0x5590bf341440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb6fa37740 4 client.16002 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.003+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3069922909 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590bf293850 con 0x5590bf33efe0 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.004+0000 7fdb5ffff640 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bf33efe0 0x5590bf341440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.004+0000 7fdb4e7fc640 10 client.16002 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.004+0000 7fdb5ffff640 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bf33efe0 0x5590bf341440 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df8603ec30cb15 server_cookie=508cc8a4535f8646 in_seq=0 out_seq=0 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/3069922909 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590bf293850 con 0x5590bf33efe0 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb4e7fc640 10 client.16002 handle_command_reply: tid=0 2024-09-16T00:46:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 client.16002 shutdown 2024-09-16T00:46:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3069922909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 msgr2=0x5590bf32af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf32af40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdb60005cf0 tx=0x7fdb600318d0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3069922909 shutdown_connections 2024-09-16T00:46:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bf33efe0 0x5590bf341440 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf32b680 0x5590bf328730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf3281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3069922909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf32af40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3069922909 >> 172.21.15.31:0/3069922909 conn(0x5590bf31ca30 msgr2=0x5590bf322710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb4e7fc640 0 client.16002 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3069922909 shutdown_connections 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3069922909 wait complete. 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.051+0000 7fdb6fa37740 20 client.16002 trim_cache size 0 max 16384 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb6fa37740 1 Processor -- start 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb6fa37740 1 -- start start 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf328fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf346d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf32b680 0x5590bf347250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681aa980 con 0x5590bf3204a0 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681ad200 con 0x5590bf320dc0 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681aaff0 con 0x5590bf32b680 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf328fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf328fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43738/0 (socket says 172.21.15.31:43738) 2024-09-16T00:46:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb677fe640 1 -- 172.21.15.31:0/3750445729 learned_addr learned my addr 172.21.15.31:0/3750445729 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.069+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf346d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.070+0000 7fdb677fe640 1 -- 172.21.15.31:0/3750445729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf32b680 msgr2=0x5590bf347250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.070+0000 7fdb677fe640 1 --2- 172.21.15.31:0/3750445729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf32b680 0x5590bf347250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.070+0000 7fdb677fe640 1 -- 172.21.15.31:0/3750445729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 msgr2=0x5590bf346d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.070+0000 7fdb677fe640 1 --2- 172.21.15.31:0/3750445729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf346d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.070+0000 7fdb677fe640 1 -- 172.21.15.31:0/3750445729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590bf320bf0 con 0x5590bf3204a0 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.070+0000 7fdb677fe640 1 --2- 172.21.15.31:0/3750445729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf328fb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdb60005e30 tx=0x7fdb60009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31deae9b5dba8d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.070+0000 7fdb5cff9640 1 -- 172.21.15.31:0/3750445729 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60032ed0 con 0x5590bf3204a0 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.070+0000 7fdb5cff9640 1 -- 172.21.15.31:0/3750445729 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb60036040 con 0x5590bf3204a0 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.070+0000 7fdb5cff9640 1 -- 172.21.15.31:0/3750445729 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb600308b0 con 0x5590bf3204a0 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3750445729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 msgr2=0x5590bf328fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3750445729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf328fb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdb60005e30 tx=0x7fdb60009ba0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3750445729 shutdown_connections 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3750445729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf32b680 0x5590bf347250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3750445729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf346d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/3750445729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf3204a0 0x5590bf328fb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3750445729 >> 172.21.15.31:0/3750445729 conn(0x5590bf31ca30 msgr2=0x5590bf3272e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3750445729 shutdown_connections 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 -- 172.21.15.31:0/3750445729 wait complete. 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 -- start start 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.071+0000 7fdb6fa37740 1 Processor -- start 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf3204a0 0x5590bf330a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf330fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb6fa37740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf32b680 0x5590bf3a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681ab6c0 con 0x5590bf32b680 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681acb30 con 0x5590bf320dc0 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb6fa37740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681abd90 con 0x5590bf3204a0 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb6fa37740 10 client.0 _ll_register_callbacks cb 0x7fdb50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf32b680 0x5590bf3a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf32b680 0x5590bf3a7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43742/0 (socket says 172.21.15.31:43742) 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb677fe640 1 -- 172.21.15.31:0/791921987 learned_addr learned my addr 172.21.15.31:0/791921987 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf330fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb677fe640 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf3204a0 0x5590bf330a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb5ffff640 1 -- 172.21.15.31:0/791921987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf3204a0 msgr2=0x5590bf330a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb5ffff640 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf3204a0 0x5590bf330a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb5ffff640 1 -- 172.21.15.31:0/791921987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf32b680 msgr2=0x5590bf3a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb5ffff640 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf32b680 0x5590bf3a7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb5ffff640 1 -- 172.21.15.31:0/791921987 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590bf3a8320 con 0x5590bf320dc0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb5ffff640 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf330fb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5590bf329830 tx=0x7fdb5001f770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d59d85375aba265 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.072+0000 7fdb4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/791921987 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50005220 con 0x5590bf320dc0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb6fa37740 1 -- 172.21.15.31:0/791921987 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb60036040 con 0x5590bf320dc0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/791921987 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fdb500053c0 con 0x5590bf320dc0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/791921987 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdb500055c0 con 0x5590bf320dc0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb6fa37740 10 client.25679 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb6fa37740 1 -- 172.21.15.31:0/791921987 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5590bf3a9670 con 0x5590bf320dc0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/791921987 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb60036040 con 0x5590bf320dc0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/791921987 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fdb50010920 con 0x5590bf320dc0 2024-09-16T00:46:29.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb6fa37740 10 client.25679 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb6fa37740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb6fa37740 20 client.25679 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bf3a9e40 0x5590bf3ac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb6fa37740 4 client.25679 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb6fa37740 1 -- 172.21.15.31:0/791921987 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5590bf28dd60 con 0x5590bf3a9e40 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.073+0000 7fdb677fe640 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bf3a9e40 0x5590bf3ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.074+0000 7fdb4e7fc640 10 client.25679 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.074+0000 7fdb677fe640 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bf3a9e40 0x5590bf3ac260 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88c6bb273b6e7b6 server_cookie=fa089fa4b033674e in_seq=0 out_seq=0 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb4e7fc640 1 -- 172.21.15.31:0/791921987 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x5590bf28dd60 con 0x5590bf3a9e40 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb4e7fc640 10 client.25679 handle_command_reply: tid=0 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 client.25679 shutdown 2024-09-16T00:46:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 -- 172.21.15.31:0/791921987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 msgr2=0x5590bf330fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:29.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf330fb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5590bf329830 tx=0x7fdb5001f770 comp rx=0 tx=0).stop 2024-09-16T00:46:29.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 -- 172.21.15.31:0/791921987 shutdown_connections 2024-09-16T00:46:29.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5590bf32b680 0x5590bf3a7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590bf320dc0 0x5590bf330fb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:46:29.373 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (42 inodes in the stack)", 2024-09-16T00:46:29.373 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:46:29.373 INFO:teuthology.orchestra.run.smithi031.stdout: "26b8bd8b-0318-46a0-8211-5666bd5eda82": { 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "26b8bd8b-0318-46a0-8211-5666bd5eda82", 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5590bf3a9e40 0x5590bf3ac260 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 --2- 172.21.15.31:0/791921987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590bf3204a0 0x5590bf330a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 -- 172.21.15.31:0/791921987 >> 172.21.15.31:0/791921987 conn(0x5590bf31ca30 msgr2=0x5590bf31b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb4e7fc640 0 client.25679 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:29.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 -- 172.21.15.31:0/791921987 shutdown_connections 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.139+0000 7fdb6fa37740 1 -- 172.21.15.31:0/791921987 wait complete. 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6fa37740 20 client.25679 trim_cache size 0 max 16384 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 -- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb540a3240 msgr2=0x7fdb540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb540a3240 0x7fdb540a5700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdb580cda70 tx=0x7fdb580d3a90 comp rx=0 tx=0).stop 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 -- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68101de0 msgr2=0x7fdb681983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68101de0 0x7fdb681983b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800e9f0 tx=0x7fdb5800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 -- 172.21.15.31:0/3462484947 shutdown_connections 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb68106f10 0x7fdb6819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdb540a3240 0x7fdb540a5700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb68102790 0x7fdb681988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 --2- 172.21.15.31:0/3462484947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb68101de0 0x7fdb681983b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 -- 172.21.15.31:0/3462484947 >> 172.21.15.31:0/3462484947 conn(0x7fdb680fd880 msgr2=0x7fdb680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 -- 172.21.15.31:0/3462484947 shutdown_connections 2024-09-16T00:46:29.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:29.140+0000 7fdb6d65e640 1 -- 172.21.15.31:0/3462484947 wait complete. 2024-09-16T00:46:30.175 INFO:tasks.cephfs.filesystem:scrub status for tag:26b8bd8b-0318-46a0-8211-5666bd5eda82 - {'path': '/', 'tag': '26b8bd8b-0318-46a0-8211-5666bd5eda82', 'options': 'recursive,force'} 2024-09-16T00:46:31.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:30 smithi081 ceph-mon[35539]: pgmap v517: 129 pgs: 129 active+clean; 569 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 263 op/s 2024-09-16T00:46:31.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:30 smithi102 ceph-mon[35364]: pgmap v517: 129 pgs: 129 active+clean; 569 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 263 op/s 2024-09-16T00:46:31.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:30 smithi031 ceph-mon[30956]: pgmap v517: 129 pgs: 129 active+clean; 569 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 263 op/s 2024-09-16T00:46:33.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:32 smithi081 ceph-mon[35539]: pgmap v518: 129 pgs: 129 active+clean; 570 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-09-16T00:46:33.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:32 smithi102 ceph-mon[35364]: pgmap v518: 129 pgs: 129 active+clean; 570 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-09-16T00:46:33.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:32 smithi031 ceph-mon[30956]: pgmap v518: 129 pgs: 129 active+clean; 570 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-09-16T00:46:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:34 smithi081 ceph-mon[35539]: pgmap v519: 129 pgs: 129 active+clean; 578 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-09-16T00:46:35.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:34 smithi102 ceph-mon[35364]: pgmap v519: 129 pgs: 129 active+clean; 578 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-09-16T00:46:35.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:34 smithi031 ceph-mon[30956]: pgmap v519: 129 pgs: 129 active+clean; 578 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-09-16T00:46:36.189 INFO:tasks.workunit.client.0.smithi031.stdout: 116 399237 174 264752 68 239583 316 2024-09-16T00:46:37.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:36 smithi081 ceph-mon[35539]: pgmap v520: 129 pgs: 129 active+clean; 588 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-09-16T00:46:37.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:36 smithi102 ceph-mon[35364]: pgmap v520: 129 pgs: 129 active+clean; 588 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-09-16T00:46:37.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:36 smithi031 ceph-mon[30956]: pgmap v520: 129 pgs: 129 active+clean; 588 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-09-16T00:46:39.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:38 smithi031 ceph-mon[30956]: pgmap v521: 129 pgs: 129 active+clean; 610 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 576 op/s 2024-09-16T00:46:39.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:38 smithi081 ceph-mon[35539]: pgmap v521: 129 pgs: 129 active+clean; 610 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 576 op/s 2024-09-16T00:46:39.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:38 smithi102 ceph-mon[35364]: pgmap v521: 129 pgs: 129 active+clean; 610 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 576 op/s 2024-09-16T00:46:41.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:40 smithi081 ceph-mon[35539]: pgmap v522: 129 pgs: 129 active+clean; 623 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 5.1 MiB/s wr, 749 op/s 2024-09-16T00:46:41.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:40 smithi102 ceph-mon[35364]: pgmap v522: 129 pgs: 129 active+clean; 623 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 5.1 MiB/s wr, 749 op/s 2024-09-16T00:46:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:40 smithi031 ceph-mon[30956]: pgmap v522: 129 pgs: 129 active+clean; 623 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 5.1 MiB/s wr, 749 op/s 2024-09-16T00:46:43.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:42 smithi081 ceph-mon[35539]: pgmap v523: 129 pgs: 129 active+clean; 625 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 5.1 MiB/s wr, 864 op/s 2024-09-16T00:46:43.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:42 smithi102 ceph-mon[35364]: pgmap v523: 129 pgs: 129 active+clean; 625 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 5.1 MiB/s wr, 864 op/s 2024-09-16T00:46:43.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:42 smithi031 ceph-mon[30956]: pgmap v523: 129 pgs: 129 active+clean; 625 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 5.1 MiB/s wr, 864 op/s 2024-09-16T00:46:45.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:44 smithi081 ceph-mon[35539]: pgmap v524: 129 pgs: 129 active+clean; 630 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 1.19k op/s 2024-09-16T00:46:45.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:44 smithi102 ceph-mon[35364]: pgmap v524: 129 pgs: 129 active+clean; 630 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 1.19k op/s 2024-09-16T00:46:45.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:44 smithi031 ceph-mon[30956]: pgmap v524: 129 pgs: 129 active+clean; 630 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 1.19k op/s 2024-09-16T00:46:46.189 INFO:tasks.workunit.client.0.smithi031.stdout: 116 47 0 51 0 40 0 2024-09-16T00:46:47.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:46 smithi081 ceph-mon[35539]: pgmap v525: 129 pgs: 129 active+clean; 621 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 1.12k op/s 2024-09-16T00:46:47.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:46 smithi102 ceph-mon[35364]: pgmap v525: 129 pgs: 129 active+clean; 621 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 1.12k op/s 2024-09-16T00:46:47.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:46 smithi031 ceph-mon[30956]: pgmap v525: 129 pgs: 129 active+clean; 621 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 1.12k op/s 2024-09-16T00:46:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a52ffd640 1 -- 172.21.15.31:0/3434847233 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f2a680a5450 con 0x55bc20244b20 2024-09-16T00:46:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a52ffd640 10 client.25597 handle_command_reply: tid=0 2024-09-16T00:46:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 client.25597 shutdown 2024-09-16T00:46:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 msgr2=0x55bc201d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201d1e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2a6402aa10 tx=0x7f2a64004040 comp rx=0 tx=0).stop 2024-09-16T00:46:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 shutdown_connections 2024-09-16T00:46:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bc201c97f0 0x55bc201c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc201c0180 0x55bc201d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55bc20244b20 0x55bc20246f40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 --2- 172.21.15.31:0/3434847233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc201bee90 0x55bc201d1e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 >> 172.21.15.31:0/3434847233 conn(0x55bc201bb0e0 msgr2=0x55bc201c05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a52ffd640 0 client.25597 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 shutdown_connections 2024-09-16T00:46:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 1 -- 172.21.15.31:0/3434847233 wait complete. 2024-09-16T00:46:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.421+0000 7f2a7c4d1740 20 client.25597 trim_cache size 0 max 16384 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 18, 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout: "f5a0ffc9": { 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 18, 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:46:47.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 145.3, 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:47.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:47.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:47.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:47.426 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:46:47.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 -- 172.21.15.31:0/1567985328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2a5c0a33d0 msgr2=0x7f2a5c0a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2a5c0a33d0 0x7f2a5c0a5890 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005fd0 tx=0x7f2a6c010740 comp rx=0 tx=0).stop 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 -- 172.21.15.31:0/1567985328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a7410d020 msgr2=0x7f2a7410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a7410d020 0x7f2a7410b870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800d770 tx=0x7f2a6800dc30 comp rx=0 tx=0).stop 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 -- 172.21.15.31:0/1567985328 shutdown_connections 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7410bdb0 0x7f2a7410c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2a5c0a33d0 0x7f2a5c0a5890 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a7410d020 0x7f2a7410b870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 --2- 172.21.15.31:0/1567985328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a74100860 0x7f2a7410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 -- 172.21.15.31:0/1567985328 >> 172.21.15.31:0/1567985328 conn(0x7f2a74077230 msgr2=0x7f2a74102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 -- 172.21.15.31:0/1567985328 shutdown_connections 2024-09-16T00:46:47.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:47.422+0000 7f2a5a7fc640 1 -- 172.21.15.31:0/1567985328 wait complete. 2024-09-16T00:46:48.243 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f5a0ffc9', quiesce took 33.6 seconds. Will release after: -8.25 2024-09-16T00:46:48.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:46:48.542 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:48.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:48 smithi031 ceph-mon[30956]: pgmap v526: 129 pgs: 129 active+clean; 618 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 1.16k op/s 2024-09-16T00:46:49.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.064+0000 7f2e54055640 1 -- 172.21.15.31:0/745337132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 msgr2=0x7f2e4c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:49.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.064+0000 7f2e54055640 1 --2- 172.21.15.31:0/745337132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 0x7f2e4c1021d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009900 tx=0x7f2e4802a410 comp rx=0 tx=0).stop 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.064+0000 7f2e54055640 1 -- 172.21.15.31:0/745337132 shutdown_connections 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.064+0000 7f2e54055640 1 --2- 172.21.15.31:0/745337132 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c106f20 0x7f2e4c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.064+0000 7f2e54055640 1 --2- 172.21.15.31:0/745337132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e4c1027a0 0x7f2e4c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.064+0000 7f2e54055640 1 --2- 172.21.15.31:0/745337132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 0x7f2e4c1021d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.064+0000 7f2e54055640 1 -- 172.21.15.31:0/745337132 >> 172.21.15.31:0/745337132 conn(0x7f2e4c0fd8f0 msgr2=0x7f2e4c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.064+0000 7f2e54055640 1 -- 172.21.15.31:0/745337132 shutdown_connections 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.064+0000 7f2e54055640 1 -- 172.21.15.31:0/745337132 wait complete. 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e54055640 1 Processor -- start 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e54055640 1 -- start start 2024-09-16T00:46:49.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e54055640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 0x7f2e4c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e54055640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e4c1027a0 0x7f2e4c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e54055640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c106f20 0x7f2e4c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e54055640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c111f80 con 0x7f2e4c101df0 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e54055640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c111e00 con 0x7f2e4c106f20 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e54055640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c112100 con 0x7f2e4c1027a0 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e51df1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 0x7f2e4c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e51df1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 0x7f2e4c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34852/0 (socket says 172.21.15.31:34852) 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e51df1640 1 -- 172.21.15.31:0/198670429 learned_addr learned my addr 172.21.15.31:0/198670429 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e525f2640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c106f20 0x7f2e4c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e515f0640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e4c1027a0 0x7f2e4c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e51df1640 1 -- 172.21.15.31:0/198670429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e4c1027a0 msgr2=0x7f2e4c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:49.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e51df1640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e4c1027a0 0x7f2e4c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e51df1640 1 -- 172.21.15.31:0/198670429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c106f20 msgr2=0x7f2e4c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e51df1640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c106f20 0x7f2e4c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.065+0000 7f2e51df1640 1 -- 172.21.15.31:0/198670429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c19f5c0 con 0x7f2e4c101df0 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.066+0000 7f2e51df1640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 0x7f2e4c19a610 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2e48006010 tx=0x7f2e48004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.066+0000 7f2e42ffd640 1 -- 172.21.15.31:0/198670429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4803f030 con 0x7f2e4c101df0 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.066+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c19f850 con 0x7f2e4c101df0 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.066+0000 7f2e42ffd640 1 -- 172.21.15.31:0/198670429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e48004440 con 0x7f2e4c101df0 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.066+0000 7f2e42ffd640 1 -- 172.21.15.31:0/198670429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48035070 con 0x7f2e4c101df0 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.066+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c1a7190 con 0x7f2e4c101df0 2024-09-16T00:46:49.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.067+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4c1021d0 con 0x7f2e4c101df0 2024-09-16T00:46:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.069+0000 7f2e42ffd640 1 -- 172.21.15.31:0/198670429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2e48042030 con 0x7f2e4c101df0 2024-09-16T00:46:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.070+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2e380a32c0 0x7f2e380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.070+0000 7f2e515f0640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2e380a32c0 0x7f2e380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.070+0000 7f2e42ffd640 1 -- 172.21.15.31:0/198670429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f2e480f02b0 con 0x7f2e4c101df0 2024-09-16T00:46:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.070+0000 7f2e515f0640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2e380a32c0 0x7f2e380a5780 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c076480 tx=0x7f2e3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.073+0000 7f2e42ffd640 1 -- 172.21.15.31:0/198670429 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e480bc690 con 0x7f2e4c101df0 2024-09-16T00:46:49.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:48 smithi081 ceph-mon[35539]: pgmap v526: 129 pgs: 129 active+clean; 618 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 1.16k op/s 2024-09-16T00:46:49.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:48 smithi102 ceph-mon[35364]: pgmap v526: 129 pgs: 129 active+clean; 618 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 1.16k op/s 2024-09-16T00:46:49.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.312+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e4c1a7360 con 0x7f2e4c101df0 2024-09-16T00:46:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.313+0000 7f2e42ffd640 1 -- 172.21.15.31:0/198670429 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f2e480c0990 con 0x7f2e4c101df0 2024-09-16T00:46:49.316 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:46:49.317 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:46:49.317 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:46:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2e380a32c0 msgr2=0x7f2e380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2e380a32c0 0x7f2e380a5780 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c076480 tx=0x7f2e3c005f90 comp rx=0 tx=0).stop 2024-09-16T00:46:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 msgr2=0x7f2e4c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 0x7f2e4c19a610 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2e48006010 tx=0x7f2e48004040 comp rx=0 tx=0).stop 2024-09-16T00:46:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 shutdown_connections 2024-09-16T00:46:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c106f20 0x7f2e4c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f2e380a32c0 0x7f2e380a5780 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e4c1027a0 0x7f2e4c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 --2- 172.21.15.31:0/198670429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c101df0 0x7f2e4c19a610 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 >> 172.21.15.31:0/198670429 conn(0x7f2e4c0fd8f0 msgr2=0x7f2e4c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 shutdown_connections 2024-09-16T00:46:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:49.316+0000 7f2e54055640 1 -- 172.21.15.31:0/198670429 wait complete. 2024-09-16T00:46:50.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:49 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/198670429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:50.178 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id f5a0ffc9 --release --await' 2024-09-16T00:46:50.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id f5a0ffc9 --release --await 2024-09-16T00:46:50.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:49 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/198670429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:50.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:49 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/198670429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:51.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:50 smithi081 ceph-mon[35539]: pgmap v527: 129 pgs: 129 active+clean; 606 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 1.04k op/s 2024-09-16T00:46:51.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:50 smithi031 ceph-mon[30956]: pgmap v527: 129 pgs: 129 active+clean; 606 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 1.04k op/s 2024-09-16T00:46:51.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:50 smithi102 ceph-mon[35364]: pgmap v527: 129 pgs: 129 active+clean; 606 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 1.04k op/s 2024-09-16T00:46:51.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.596+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/905547870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d4100e20 msgr2=0x7fa9d4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.596+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/905547870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d4100e20 0x7fa9d4101200 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009900 tx=0x7fa9c402a480 comp rx=0 tx=0).stop 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.596+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/905547870 shutdown_connections 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.596+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/905547870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4102ef0 0x7fa9d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.596+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/905547870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9d4102570 0x7fa9d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.596+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/905547870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d4100e20 0x7fa9d4101200 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.596+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/905547870 >> 172.21.15.31:0/905547870 conn(0x7fa9d40fc7d0 msgr2=0x7fa9d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.596+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/905547870 shutdown_connections 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.596+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/905547870 wait complete. 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d9bdb640 1 Processor -- start 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d9bdb640 1 -- start start 2024-09-16T00:46:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d9bdb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4100e20 0x7fa9d41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d9bdb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d4102570 0x7fa9d4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d9bdb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9d4102ef0 0x7fa9d41a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d9bdb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4110fa0 con 0x7fa9d4102570 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d9bdb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4110e20 con 0x7fa9d4102ef0 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4100e20 0x7fa9d41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4100e20 0x7fa9d41992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:45778/0 (socket says 172.21.15.31:45778) 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1113089841 learned_addr learned my addr 172.21.15.31:0/1113089841 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9cbfff640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d4102570 0x7fa9d4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/1113089841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4111120 con 0x7fa9d4100e20 2024-09-16T00:46:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.597+0000 7fa9d3fff640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9d4102ef0 0x7fa9d41a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1113089841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9d4102ef0 msgr2=0x7fa9d41a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9d4102ef0 0x7fa9d41a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d4102570 msgr2=0x7fa9d4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d4102570 0x7fa9d4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1113089841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d41a48b0 con 0x7fa9d4100e20 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4100e20 0x7fa9d41992f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa9c402aa70 tx=0x7fa9c4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d17fa640 1 -- 172.21.15.31:0/1113089841 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c403f070 con 0x7fa9d4100e20 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/1113089841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d41a4a80 con 0x7fa9d4100e20 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d17fa640 1 -- 172.21.15.31:0/1113089841 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c4031c30 con 0x7fa9d4100e20 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d17fa640 1 -- 172.21.15.31:0/1113089841 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4035030 con 0x7fa9d4100e20 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.598+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/1113089841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d41a4f60 con 0x7fa9d4100e20 2024-09-16T00:46:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.601+0000 7fa9d17fa640 1 -- 172.21.15.31:0/1113089841 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa9c4042070 con 0x7fa9d4100e20 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.601+0000 7fa9dbfb4740 1 Processor -- start 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.601+0000 7fa9dbfb4740 1 -- start start 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b060 0x55a3c643b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 0x55a3c6445d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c6446240 0x55a3c6448630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a3c63a0620 con 0x55a3c6446240 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a3c6347e50 con 0x55a3c643b980 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a3c639c5e0 con 0x55a3c643b060 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 0x55a3c6445d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c6446240 0x55a3c6448630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b060 0x55a3c643b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d17fa640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9c00a3240 0x7fa9c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b060 0x55a3c643b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:45782/0 (socket says 172.21.15.31:45782) 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d3fff640 1 -- 172.21.15.31:0/1231873610 learned_addr learned my addr 172.21.15.31:0/1231873610 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d3fff640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9c00a3240 0x7fa9c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d3fff640 1 -- 172.21.15.31:0/1231873610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 msgr2=0x55a3c6445d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d3fff640 1 --2- 172.21.15.31:0/1231873610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 0x55a3c6445d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d3fff640 1 -- 172.21.15.31:0/1231873610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c6446240 msgr2=0x55a3c6448630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d3fff640 1 --2- 172.21.15.31:0/1231873610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c6446240 0x55a3c6448630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d3fff640 1 -- 172.21.15.31:0/1231873610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c6338d40 con 0x55a3c643b060 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.602+0000 7fa9d17fa640 1 -- 172.21.15.31:0/1113089841 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa9c40f0300 con 0x7fa9d4100e20 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.603+0000 7fa9d3fff640 1 --2- 172.21.15.31:0/1231873610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b060 0x55a3c643b440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc005c40 tx=0x7fa9cc013f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6661ab861302d7ff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.603+0000 7fa9d3fff640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9c00a3240 0x7fa9c00a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc01d000 tx=0x7fa9cc01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.603+0000 7fa9c9ffb640 1 -- 172.21.15.31:0/1231873610 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00fde0 con 0x55a3c643b060 2024-09-16T00:46:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.603+0000 7fa9c9ffb640 1 -- 172.21.15.31:0/1231873610 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9cc022020 con 0x55a3c643b060 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.603+0000 7fa9c9ffb640 1 -- 172.21.15.31:0/1231873610 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc010690 con 0x55a3c643b060 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1231873610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b060 msgr2=0x55a3c643b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1231873610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b060 0x55a3c643b440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc005c40 tx=0x7fa9cc013f50 comp rx=0 tx=0).stop 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1231873610 shutdown_connections 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1231873610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c6446240 0x55a3c6448630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1231873610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 0x55a3c6445d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1231873610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b060 0x55a3c643b440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1231873610 >> 172.21.15.31:0/1231873610 conn(0x55a3c64375f0 msgr2=0x55a3c64379d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1231873610 shutdown_connections 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1231873610 wait complete. 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.604+0000 7fa9dbfb4740 1 -- start start 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9dbfb4740 1 Processor -- start 2024-09-16T00:46:51.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c6443110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 0x55a3c6443650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c6446240 0x55a3c6443b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a3c6448e60 con 0x55a3c643b060 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a3c6328f10 con 0x55a3c643b980 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a3c6448fe0 con 0x55a3c6446240 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9dbfb4740 10 client.0 _ll_register_callbacks cb 0x55a3c634cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c6443110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9cbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 0x55a3c6443650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c6443110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34892/0 (socket says 172.21.15.31:34892) 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1403442581 learned_addr learned my addr 172.21.15.31:0/1403442581 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9d3fff640 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c6446240 0x55a3c6443b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1403442581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c6446240 msgr2=0x55a3c6443b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:51.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c6446240 0x55a3c6443b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1403442581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 msgr2=0x55a3c6443650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 0x55a3c6443650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1403442581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c60af580 con 0x55a3c643b060 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.605+0000 7fa9cbfff640 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 0x55a3c6443650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c6443110 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40bc2c0 tx=0x7fa9c402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d6b5c0a3135b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1403442581 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4049070 con 0x55a3c643b060 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1403442581 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa9c40c4cf0 con 0x55a3c643b060 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1403442581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9cc022020 con 0x55a3c643b060 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1403442581 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9c40b8860 con 0x55a3c643b060 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9dbfb4740 10 client.16026 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1403442581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a3c6179960 con 0x55a3c643b060 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1403442581 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9cc022020 con 0x55a3c643b060 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1403442581 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fa9c40bba10 con 0x55a3c643b060 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9dbfb4740 10 client.16026 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9dbfb4740 10 client.16026 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9dbfb4740 20 client.16026 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a3c6455d10 0x55a3c6458160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9dbfb4740 4 client.16026 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1403442581 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3c63adb90 con 0x55a3c6455d10 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.606+0000 7fa9cbfff640 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a3c6455d10 0x55a3c6458160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.607+0000 7fa9cbfff640 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a3c6455d10 0x55a3c6458160 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae32af14ca411af2 server_cookie=d959c328c2d3d44a in_seq=0 out_seq=0 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.607+0000 7fa9b2ffd640 10 client.16026 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1403442581 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3c63adb90 con 0x55a3c6455d10 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9b2ffd640 10 client.16026 handle_command_reply: tid=0 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 client.16026 shutdown 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1403442581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 msgr2=0x55a3c6443110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c6443110 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40bc2c0 tx=0x7fa9c402a480 comp rx=0 tx=0).stop 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1403442581 shutdown_connections 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a3c6455d10 0x55a3c6458160 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c6446240 0x55a3c6443b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b980 0x55a3c6443650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1403442581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c6443110 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9b2ffd640 0 client.16026 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1403442581 >> 172.21.15.31:0/1403442581 conn(0x55a3c64375f0 msgr2=0x55a3c643d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1403442581 shutdown_connections 2024-09-16T00:46:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.609+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1403442581 wait complete. 2024-09-16T00:46:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.610+0000 7fa9dbfb4740 20 client.16026 trim_cache size 0 max 16384 2024-09-16T00:46:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9dbfb4740 1 Processor -- start 2024-09-16T00:46:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9dbfb4740 1 -- start start 2024-09-16T00:46:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c64375f0 0x55a3c643db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c643e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 0x55a3c6436640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41aaa10 con 0x55a3c643b060 2024-09-16T00:46:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41ad290 con 0x55a3c64375f0 2024-09-16T00:46:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41ab080 con 0x55a3c643b980 2024-09-16T00:46:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c643e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c64375f0 0x55a3c643db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c643e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34906/0 (socket says 172.21.15.31:34906) 2024-09-16T00:46:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9d3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 0x55a3c6436640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9cbfff640 1 -- 172.21.15.31:0/2190990609 learned_addr learned my addr 172.21.15.31:0/2190990609 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9d37fe640 1 -- 172.21.15.31:0/2190990609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 msgr2=0x55a3c6436640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/2190990609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 0x55a3c6436640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9d37fe640 1 -- 172.21.15.31:0/2190990609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 msgr2=0x55a3c643e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/2190990609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c643e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9d37fe640 1 -- 172.21.15.31:0/2190990609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c6445050 con 0x55a3c64375f0 2024-09-16T00:46:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.629+0000 7fa9d3fff640 1 --2- 172.21.15.31:0/2190990609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 0x55a3c6436640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.630+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/2190990609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c64375f0 0x55a3c643db10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40b8a20 tx=0x7fa9c4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ecf718e6f141621 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.630+0000 7fa9c97fa640 1 -- 172.21.15.31:0/2190990609 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4049030 con 0x55a3c64375f0 2024-09-16T00:46:51.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.630+0000 7fa9c97fa640 1 -- 172.21.15.31:0/2190990609 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c403b460 con 0x55a3c64375f0 2024-09-16T00:46:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.630+0000 7fa9c97fa640 1 -- 172.21.15.31:0/2190990609 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40b8310 con 0x55a3c64375f0 2024-09-16T00:46:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/2190990609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c64375f0 msgr2=0x55a3c643db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/2190990609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c64375f0 0x55a3c643db10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40b8a20 tx=0x7fa9c4009260 comp rx=0 tx=0).stop 2024-09-16T00:46:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/2190990609 shutdown_connections 2024-09-16T00:46:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/2190990609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 0x55a3c6436640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/2190990609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c643b060 0x55a3c643e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/2190990609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c64375f0 0x55a3c643db10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/2190990609 >> 172.21.15.31:0/2190990609 conn(0x55a3c643eb00 msgr2=0x55a3c6435f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:51.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/2190990609 shutdown_connections 2024-09-16T00:46:51.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/2190990609 wait complete. 2024-09-16T00:46:51.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 -- start start 2024-09-16T00:46:51.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.631+0000 7fa9dbfb4740 1 Processor -- start 2024-09-16T00:46:51.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c64375f0 0x55a3c644a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b060 0x55a3c644a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9dbfb4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 0x55a3c644abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41ab750 con 0x55a3c64375f0 2024-09-16T00:46:51.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41acbc0 con 0x55a3c643b060 2024-09-16T00:46:51.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9dbfb4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41abe20 con 0x55a3c643b980 2024-09-16T00:46:51.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9dbfb4740 10 client.0 _ll_register_callbacks cb 0x7fa9c40bbf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:51.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c64375f0 0x55a3c644a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c64375f0 0x55a3c644a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34922/0 (socket says 172.21.15.31:34922) 2024-09-16T00:46:51.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1135001860 learned_addr learned my addr 172.21.15.31:0/1135001860 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:51.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d3fff640 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 0x55a3c644abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9cbfff640 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b060 0x55a3c644a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1135001860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 msgr2=0x55a3c644abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:51.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 0x55a3c644abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1135001860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b060 msgr2=0x55a3c644a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:51.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b060 0x55a3c644a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d37fe640 1 -- 172.21.15.31:0/1135001860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c64bfba0 con 0x55a3c64375f0 2024-09-16T00:46:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9d37fe640 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c64375f0 0x55a3c644a130 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4005bf0 tx=0x7fa9c40c4f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416a5f275771bff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.632+0000 7fa9b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:46:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1135001860 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40d4030 con 0x55a3c64375f0 2024-09-16T00:46:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1135001860 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa9c4031960 con 0x55a3c64375f0 2024-09-16T00:46:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1135001860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a3c6432950 con 0x55a3c64375f0 2024-09-16T00:46:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1135001860 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9c4049070 con 0x55a3c64375f0 2024-09-16T00:46:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9dbfb4740 10 client.16032 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1135001860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a3c64bfba0 con 0x55a3c64375f0 2024-09-16T00:46:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1135001860 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a3c6432950 con 0x55a3c64375f0 2024-09-16T00:46:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1135001860 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fa9c40d5390 con 0x55a3c64375f0 2024-09-16T00:46:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9dbfb4740 10 client.16032 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9dbfb4740 10 client.16032 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:46:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9dbfb4740 20 client.16032 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a3c6446240 0x55a3c64c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9dbfb4740 4 client.16032 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f5a0ffc9", "release": true, "await": true}] 2024-09-16T00:46:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1135001860 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f5a0ffc9", "release": true, "await": true}) -- 0x55a3c63a80a0 con 0x55a3c6446240 2024-09-16T00:46:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.633+0000 7fa9cbfff640 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a3c6446240 0x55a3c64c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.634+0000 7fa9b2ffd640 10 client.16032 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:51.634+0000 7fa9cbfff640 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a3c6446240 0x55a3c64c31d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c03d4254ee88e3 server_cookie=7656903480d24e4e in_seq=0 out_seq=0 2024-09-16T00:46:52.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9b2ffd640 1 -- 172.21.15.31:0/1135001860 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a3c63a80a0 con 0x55a3c6446240 2024-09-16T00:46:52.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9b2ffd640 10 client.16032 handle_command_reply: tid=0 2024-09-16T00:46:52.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 client.16032 shutdown 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1135001860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c64375f0 msgr2=0x55a3c644a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c64375f0 0x55a3c644a130 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4005bf0 tx=0x7fa9c40c4f20 comp rx=0 tx=0).stop 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1135001860 shutdown_connections 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a3c6446240 0x55a3c64c31d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a3c643b980 0x55a3c644abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c643b060 0x55a3c644a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9b2ffd640 0 client.16032 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 --2- 172.21.15.31:0/1135001860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3c64375f0 0x55a3c644a130 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1135001860 >> 172.21.15.31:0/1135001860 conn(0x55a3c643eb00 msgr2=0x55a3c64438f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1135001860 shutdown_connections 2024-09-16T00:46:52.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 1 -- 172.21.15.31:0/1135001860 wait complete. 2024-09-16T00:46:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.686+0000 7fa9dbfb4740 20 client.16032 trim_cache size 0 max 16384 2024-09-16T00:46:52.689 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:46:52.689 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:46:52.689 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:46:52.689 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 20, 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "f5a0ffc9": { 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 20, 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 145.3, 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:46:52.690 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:46:52.691 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:46:52.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:46:52.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:46:52.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:52.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:52.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:52.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:52.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:46:52.691 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.687+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9c00a3240 msgr2=0x7fa9c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.687+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9c00a3240 0x7fa9c00a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc01d000 tx=0x7fa9cc01f000 comp rx=0 tx=0).stop 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.687+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/1113089841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4100e20 msgr2=0x7fa9d41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.687+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4100e20 0x7fa9d41992f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa9c402aa70 tx=0x7fa9c4003c10 comp rx=0 tx=0).stop 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.688+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/1113089841 shutdown_connections 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.688+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9d4102ef0 0x7fa9d41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.688+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9d4102570 0x7fa9d4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.688+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa9c00a3240 0x7fa9c00a5700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.688+0000 7fa9d9bdb640 1 --2- 172.21.15.31:0/1113089841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4100e20 0x7fa9d41992f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.688+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/1113089841 >> 172.21.15.31:0/1113089841 conn(0x7fa9d40fc7d0 msgr2=0x7fa9d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:52.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.688+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/1113089841 shutdown_connections 2024-09-16T00:46:52.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:52.688+0000 7fa9d9bdb640 1 -- 172.21.15.31:0/1113089841 wait complete. 2024-09-16T00:46:53.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:52 smithi031 ceph-mon[30956]: pgmap v528: 129 pgs: 129 active+clean; 608 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 1k op/s 2024-09-16T00:46:53.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:52 smithi081 ceph-mon[35539]: pgmap v528: 129 pgs: 129 active+clean; 608 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 1k op/s 2024-09-16T00:46:53.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:52 smithi102 ceph-mon[35364]: pgmap v528: 129 pgs: 129 active+clean; 608 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 1k op/s 2024-09-16T00:46:54.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:53 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:46:54.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:53 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:46:54.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:53 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:46:54.655 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f5a0ffc9', total seconds elapsed: 40.0 2024-09-16T00:46:54.656 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-16T00:46:54.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:46:54.996 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:55.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:54 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:46:55.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:54 smithi081 ceph-mon[35539]: pgmap v529: 129 pgs: 129 active+clean; 608 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 1.04k op/s 2024-09-16T00:46:55.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:54 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:46:55.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:54 smithi031 ceph-mon[30956]: pgmap v529: 129 pgs: 129 active+clean; 608 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 1.04k op/s 2024-09-16T00:46:55.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:54 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:46:55.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:54 smithi102 ceph-mon[35364]: pgmap v529: 129 pgs: 129 active+clean; 608 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 1.04k op/s 2024-09-16T00:46:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.484+0000 7f42cf576640 1 -- 172.21.15.31:0/3662370269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 msgr2=0x7f42c8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.484+0000 7f42cf576640 1 --2- 172.21.15.31:0/3662370269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 0x7f42c8103110 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009930 tx=0x7f42c402a420 comp rx=0 tx=0).stop 2024-09-16T00:46:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.484+0000 7f42cf576640 1 -- 172.21.15.31:0/3662370269 shutdown_connections 2024-09-16T00:46:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.484+0000 7f42cf576640 1 --2- 172.21.15.31:0/3662370269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42c81082a0 0x7f42c810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.484+0000 7f42cf576640 1 --2- 172.21.15.31:0/3662370269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42c81036e0 0x7f42c8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.484+0000 7f42cf576640 1 --2- 172.21.15.31:0/3662370269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 0x7f42c8103110 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.484+0000 7f42cf576640 1 -- 172.21.15.31:0/3662370269 >> 172.21.15.31:0/3662370269 conn(0x7f42c80fc7d0 msgr2=0x7f42c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.484+0000 7f42cf576640 1 -- 172.21.15.31:0/3662370269 shutdown_connections 2024-09-16T00:46:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.484+0000 7f42cf576640 1 -- 172.21.15.31:0/3662370269 wait complete. 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cf576640 1 Processor -- start 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cf576640 1 -- start start 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cf576640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 0x7f42c81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cf576640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42c81036e0 0x7f42c8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cf576640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42c81082a0 0x7f42c819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cf576640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f42c81120a0 con 0x7f42c8102d30 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cf576640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f42c8111f20 con 0x7f42c81036e0 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cf576640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f42c8112220 con 0x7f42c81082a0 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cd312640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 0x7f42c81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cd312640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 0x7f42c81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34948/0 (socket says 172.21.15.31:34948) 2024-09-16T00:46:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cd312640 1 -- 172.21.15.31:0/129735795 learned_addr learned my addr 172.21.15.31:0/129735795 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42ccb11640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42c81036e0 0x7f42c8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.485+0000 7f42cdb13640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42c81082a0 0x7f42c819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.486+0000 7f42cdb13640 1 -- 172.21.15.31:0/129735795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42c81036e0 msgr2=0x7f42c8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.486+0000 7f42cdb13640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42c81036e0 0x7f42c8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.486+0000 7f42cdb13640 1 -- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 msgr2=0x7f42c81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.486+0000 7f42cdb13640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 0x7f42c81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.486+0000 7f42cdb13640 1 -- 172.21.15.31:0/129735795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c819d570 con 0x7f42c81082a0 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.486+0000 7f42cd312640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 0x7f42c81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.486+0000 7f42cdb13640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42c81082a0 0x7f42c819ce90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42c000ec50 tx=0x7f42c000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.487+0000 7f42be7fc640 1 -- 172.21.15.31:0/129735795 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0003b70 con 0x7f42c81082a0 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.487+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c819d860 con 0x7f42c81082a0 2024-09-16T00:46:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.487+0000 7f42be7fc640 1 -- 172.21.15.31:0/129735795 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42c0003d10 con 0x7f42c81082a0 2024-09-16T00:46:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.487+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c81a5140 con 0x7f42c81082a0 2024-09-16T00:46:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.487+0000 7f42be7fc640 1 -- 172.21.15.31:0/129735795 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0010890 con 0x7f42c81082a0 2024-09-16T00:46:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.488+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c806a850 con 0x7f42c81082a0 2024-09-16T00:46:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.489+0000 7f42be7fc640 1 -- 172.21.15.31:0/129735795 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f42c0003ec0 con 0x7f42c81082a0 2024-09-16T00:46:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.489+0000 7f42be7fc640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f42b40a32c0 0x7f42b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.490+0000 7f42cd312640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f42b40a32c0 0x7f42b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.490+0000 7f42be7fc640 1 -- 172.21.15.31:0/129735795 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f42c00d0190 con 0x7f42c81082a0 2024-09-16T00:46:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.490+0000 7f42cd312640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f42b40a32c0 0x7f42b40a5780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000c00 tx=0x7f42c4009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.494+0000 7f42be7fc640 1 -- 172.21.15.31:0/129735795 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42c009caa0 con 0x7f42c81082a0 2024-09-16T00:46:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.723+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42c810a2b0 con 0x7f42c81082a0 2024-09-16T00:46:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.724+0000 7f42be7fc640 1 -- 172.21.15.31:0/129735795 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f42c009c3a0 con 0x7f42c81082a0 2024-09-16T00:46:55.726 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:46:55.726 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:46:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:46:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f42b40a32c0 msgr2=0x7f42b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f42b40a32c0 0x7f42b40a5780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000c00 tx=0x7f42c4009560 comp rx=0 tx=0).stop 2024-09-16T00:46:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42c81082a0 msgr2=0x7f42c819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42c81082a0 0x7f42c819ce90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42c000ec50 tx=0x7f42c000c6a0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 shutdown_connections 2024-09-16T00:46:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f42b40a32c0 0x7f42b40a5780 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42c81082a0 0x7f42c819ce90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42c81036e0 0x7f42c8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 --2- 172.21.15.31:0/129735795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42c8102d30 0x7f42c81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.726+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 >> 172.21.15.31:0/129735795 conn(0x7f42c80fc7d0 msgr2=0x7f42c8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.727+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 shutdown_connections 2024-09-16T00:46:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:55.727+0000 7f42cf576640 1 -- 172.21.15.31:0/129735795 wait complete. 2024-09-16T00:46:56.190 INFO:tasks.workunit.client.0.smithi031.stdout: 116 7669 67 4797 15 4813 104 2024-09-16T00:46:56.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:55 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/129735795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:56.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:55 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/129735795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:56.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:55 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/129735795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:46:56.502 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-16T00:46:56.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-16T00:46:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:46:57.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:56 smithi081 ceph-mon[35539]: pgmap v530: 129 pgs: 129 active+clean; 610 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 955 KiB/s wr, 741 op/s 2024-09-16T00:46:57.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:56 smithi031 ceph-mon[30956]: pgmap v530: 129 pgs: 129 active+clean; 610 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 955 KiB/s wr, 741 op/s 2024-09-16T00:46:57.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:56 smithi102 ceph-mon[35364]: pgmap v530: 129 pgs: 129 active+clean; 610 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 955 KiB/s wr, 741 op/s 2024-09-16T00:46:57.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.349+0000 7f9636c9f640 1 -- 172.21.15.31:0/2067689673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102570 msgr2=0x7f96301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.349+0000 7f9636c9f640 1 --2- 172.21.15.31:0/2067689673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102570 0x7f96301029b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f961c009900 tx=0x7f961c02a480 comp rx=0 tx=0).stop 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 -- 172.21.15.31:0/2067689673 shutdown_connections 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 --2- 172.21.15.31:0/2067689673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9630102ef0 0x7f963010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 --2- 172.21.15.31:0/2067689673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102570 0x7f96301029b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 --2- 172.21.15.31:0/2067689673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9630100e20 0x7f9630101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 -- 172.21.15.31:0/2067689673 >> 172.21.15.31:0/2067689673 conn(0x7f96300fc7d0 msgr2=0x7f96300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 -- 172.21.15.31:0/2067689673 shutdown_connections 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 -- 172.21.15.31:0/2067689673 wait complete. 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 Processor -- start 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 -- start start 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9630100e20 0x7f9630198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9630102570 0x7f9630198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102ef0 0x7f963019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.350+0000 7f9636c9f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9630110fd0 con 0x7f9630102ef0 2024-09-16T00:46:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f9636c9f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9630110e50 con 0x7f9630102570 2024-09-16T00:46:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f9636c9f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9630111150 con 0x7f9630100e20 2024-09-16T00:46:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f963523c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102ef0 0x7f963019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f962ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9630102570 0x7f9630198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f9634a3b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9630100e20 0x7f9630198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f963523c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102ef0 0x7f963019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52204/0 (socket says 172.21.15.31:52204) 2024-09-16T00:46:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f963523c640 1 -- 172.21.15.31:0/1923688176 learned_addr learned my addr 172.21.15.31:0/1923688176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f962ffff640 1 -- 172.21.15.31:0/1923688176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9630100e20 msgr2=0x7f9630198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f962ffff640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9630100e20 0x7f9630198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f962ffff640 1 -- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102ef0 msgr2=0x7f963019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f962ffff640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102ef0 0x7f963019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f962ffff640 1 -- 172.21.15.31:0/1923688176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963019d4f0 con 0x7f9630102570 2024-09-16T00:46:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f963523c640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102ef0 0x7f963019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f9634a3b640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9630100e20 0x7f9630198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.351+0000 7f962ffff640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9630102570 0x7f9630198a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f961c02a9a0 tx=0x7f961c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.352+0000 7f962dffb640 1 -- 172.21.15.31:0/1923688176 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c03f070 con 0x7f9630102570 2024-09-16T00:46:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.352+0000 7f9636c9f640 1 -- 172.21.15.31:0/1923688176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963019d780 con 0x7f9630102570 2024-09-16T00:46:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.352+0000 7f962dffb640 1 -- 172.21.15.31:0/1923688176 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961c031c20 con 0x7f9630102570 2024-09-16T00:46:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.352+0000 7f962dffb640 1 -- 172.21.15.31:0/1923688176 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c035030 con 0x7f9630102570 2024-09-16T00:46:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.352+0000 7f9636c9f640 1 -- 172.21.15.31:0/1923688176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96301a50c0 con 0x7f9630102570 2024-09-16T00:46:57.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.354+0000 7f962dffb640 1 -- 172.21.15.31:0/1923688176 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f961c042070 con 0x7f9630102570 2024-09-16T00:46:57.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f9639078740 1 Processor -- start 2024-09-16T00:46:57.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f9639078740 1 -- start start 2024-09-16T00:46:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f9639078740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08404bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f9639078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 0x55a08405a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f962dffb640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96200a3240 0x7f96200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f9639078740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a08405d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f9639078740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a083fb0160 con 0x55a08405ae40 2024-09-16T00:46:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f9639078740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a083f57720 con 0x55a08404b730 2024-09-16T00:46:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f9639078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a083fac120 con 0x55a08404c050 2024-09-16T00:46:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f963523c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08404bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f963523c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08404bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:41212/0 (socket says 172.21.15.31:41212) 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f963523c640 1 -- 172.21.15.31:0/3398587103 learned_addr learned my addr 172.21.15.31:0/3398587103 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f9634a3b640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96200a3240 0x7f96200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f962ffff640 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 0x55a08405a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f963523c640 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a08405d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.355+0000 7f962dffb640 1 -- 172.21.15.31:0/1923688176 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f961c0eff40 con 0x7f9630102570 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f963523c640 1 -- 172.21.15.31:0/3398587103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 msgr2=0x55a08405a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f963523c640 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 0x55a08405a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f963523c640 1 -- 172.21.15.31:0/3398587103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 msgr2=0x55a08405d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f963523c640 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a08405d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f963523c640 1 -- 172.21.15.31:0/3398587103 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a083d21ba0 con 0x55a08404b730 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f962ffff640 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 0x55a08405a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f963523c640 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08404bb10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9628005c40 tx=0x7f96280113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ac8efebb0ee028 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f9634a3b640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96200a3240 0x7f96200a5700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9624009fc0 tx=0x7f9624005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f961affd640 1 -- 172.21.15.31:0/3398587103 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628011ce0 con 0x55a08404b730 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f961affd640 1 -- 172.21.15.31:0/3398587103 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9628011e80 con 0x55a08404b730 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.356+0000 7f961affd640 1 -- 172.21.15.31:0/3398587103 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962800e560 con 0x55a08404b730 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.357+0000 7f9639078740 1 -- 172.21.15.31:0/3398587103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 msgr2=0x55a08404bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.357+0000 7f9639078740 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08404bb10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9628005c40 tx=0x7f96280113e0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.357+0000 7f9639078740 1 -- 172.21.15.31:0/3398587103 shutdown_connections 2024-09-16T00:46:57.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.357+0000 7f9639078740 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a08405d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.357+0000 7f9639078740 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 0x55a08405a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.357+0000 7f9639078740 1 --2- 172.21.15.31:0/3398587103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08404bb10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.357+0000 7f9639078740 1 -- 172.21.15.31:0/3398587103 >> 172.21.15.31:0/3398587103 conn(0x55a084047de0 msgr2=0x55a0840481c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:57.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.357+0000 7f9639078740 1 -- 172.21.15.31:0/3398587103 shutdown_connections 2024-09-16T00:46:57.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.357+0000 7f9639078740 1 -- 172.21.15.31:0/3398587103 wait complete. 2024-09-16T00:46:57.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9639078740 1 -- start start 2024-09-16T00:46:57.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9639078740 1 Processor -- start 2024-09-16T00:46:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9639078740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08404b730 0x55a084057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9639078740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a0840582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9639078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08405ae40 0x55a084058810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9639078740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a08405da10 con 0x55a08404b730 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9639078740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a083f38900 con 0x55a08404c050 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9639078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a08405db90 con 0x55a08405ae40 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9634a3b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08404b730 0x55a084057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9639078740 10 client.0 _ll_register_callbacks cb 0x55a083f36780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9634a3b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08404b730 0x55a084057d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52222/0 (socket says 172.21.15.31:52222) 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f9634a3b640 1 -- 172.21.15.31:0/1414206109 learned_addr learned my addr 172.21.15.31:0/1414206109 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f963523c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08405ae40 0x55a084058810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.358+0000 7f962ffff640 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a0840582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f9634a3b640 1 -- 172.21.15.31:0/1414206109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08405ae40 msgr2=0x55a084058810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f9634a3b640 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08405ae40 0x55a084058810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f9634a3b640 1 -- 172.21.15.31:0/1414206109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 msgr2=0x55a0840582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:57.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f9634a3b640 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a0840582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f9634a3b640 1 -- 172.21.15.31:0/1414206109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a083f78c50 con 0x55a08404b730 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f963523c640 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08405ae40 0x55a084058810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f962ffff640 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a0840582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f9634a3b640 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08404b730 0x55a084057d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9624003730 tx=0x7f9624004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b380a026b537ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f960ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f960ffff640 1 -- 172.21.15.31:0/1414206109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962400f870 con 0x55a08404b730 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f960ffff640 1 -- 172.21.15.31:0/1414206109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f962400fa10 con 0x55a08404b730 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.359+0000 7f9639078740 1 -- 172.21.15.31:0/1414206109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a083fbd890 con 0x55a08404b730 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f960ffff640 1 -- 172.21.15.31:0/1414206109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f962400fc10 con 0x55a08404b730 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f9639078740 10 client.16050 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f9639078740 1 -- 172.21.15.31:0/1414206109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a0840697b0 con 0x55a08404b730 2024-09-16T00:46:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f960ffff640 1 -- 172.21.15.31:0/1414206109 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a083fbd890 con 0x55a08404b730 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f960ffff640 1 -- 172.21.15.31:0/1414206109 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f962401a070 con 0x55a08404b730 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f9639078740 10 client.16050 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f9639078740 10 client.16050 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f9639078740 20 client.16050 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f9639078740 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a08406ac10 0x55a08406d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f9639078740 4 client.16050 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f9639078740 1 -- 172.21.15.31:0/1414206109 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a083fb7be0 con 0x55a08406ac10 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.360+0000 7f962ffff640 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a08406ac10 0x55a08406d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.361+0000 7f960ffff640 10 client.16050 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.361+0000 7f962ffff640 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a08406ac10 0x55a08406d080 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70fc915d676699ab server_cookie=4900ee55943bf6b2 in_seq=0 out_seq=0 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f960ffff640 1 -- 172.21.15.31:0/1414206109 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a083fb7be0 con 0x55a08406ac10 2024-09-16T00:46:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f960ffff640 10 client.16050 handle_command_reply: tid=0 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 client.16050 shutdown 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 -- 172.21.15.31:0/1414206109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08404b730 msgr2=0x55a084057d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08404b730 0x55a084057d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9624003730 tx=0x7f9624004730 comp rx=0 tx=0).stop 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 -- 172.21.15.31:0/1414206109 shutdown_connections 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a08406ac10 0x55a08406d080 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08405ae40 0x55a084058810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a0840582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f960ffff640 0 client.16050 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 --2- 172.21.15.31:0/1414206109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08404b730 0x55a084057d90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 -- 172.21.15.31:0/1414206109 >> 172.21.15.31:0/1414206109 conn(0x55a084047de0 msgr2=0x55a08404db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 -- 172.21.15.31:0/1414206109 shutdown_connections 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 1 -- 172.21.15.31:0/1414206109 wait complete. 2024-09-16T00:46:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.363+0000 7f9639078740 20 client.16050 trim_cache size 0 max 16384 2024-09-16T00:46:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.381+0000 7f9639078740 1 Processor -- start 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.381+0000 7f9639078740 1 -- start start 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f9639078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404b730 0x55a08404dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f9639078740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a084072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f9639078740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a084072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f9639078740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96301aaaf0 con 0x55a08405ae40 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f9639078740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f96301ad370 con 0x55a08404c050 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f9639078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f96301ab160 con 0x55a08404b730 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f963523c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a084072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f962ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a084072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f963523c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a084072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52224/0 (socket says 172.21.15.31:52224) 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f962ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a084072950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:41218/0 (socket says 172.21.15.31:41218) 2024-09-16T00:46:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f963523c640 1 -- 172.21.15.31:0/2067819422 learned_addr learned my addr 172.21.15.31:0/2067819422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f9634a3b640 1 --2- 172.21.15.31:0/2067819422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404b730 0x55a08404dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f962ffff640 1 -- 172.21.15.31:0/2067819422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404b730 msgr2=0x55a08404dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f962ffff640 1 --2- 172.21.15.31:0/2067819422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404b730 0x55a08404dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f962ffff640 1 -- 172.21.15.31:0/2067819422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 msgr2=0x55a084072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f962ffff640 1 --2- 172.21.15.31:0/2067819422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a084072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f962ffff640 1 -- 172.21.15.31:0/2067819422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a084056d40 con 0x55a08404c050 2024-09-16T00:46:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.382+0000 7f962ffff640 1 --2- 172.21.15.31:0/2067819422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a084072950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b8660 tx=0x7f961c036450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13487b9aaa341f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:57.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f961a7fc640 1 -- 172.21.15.31:0/2067819422 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c049040 con 0x55a08404c050 2024-09-16T00:46:57.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f961a7fc640 1 -- 172.21.15.31:0/2067819422 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961c0c4dc0 con 0x55a08404c050 2024-09-16T00:46:57.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f961a7fc640 1 -- 172.21.15.31:0/2067819422 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c03b460 con 0x55a08404c050 2024-09-16T00:46:57.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f9639078740 1 -- 172.21.15.31:0/2067819422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 msgr2=0x55a084072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:57.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f9639078740 1 --2- 172.21.15.31:0/2067819422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a084072950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b8660 tx=0x7f961c036450 comp rx=0 tx=0).stop 2024-09-16T00:46:57.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f9639078740 1 -- 172.21.15.31:0/2067819422 shutdown_connections 2024-09-16T00:46:57.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f9639078740 1 --2- 172.21.15.31:0/2067819422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a084072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f9639078740 1 --2- 172.21.15.31:0/2067819422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404c050 0x55a084072950 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f9639078740 1 --2- 172.21.15.31:0/2067819422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404b730 0x55a08404dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f9639078740 1 -- 172.21.15.31:0/2067819422 >> 172.21.15.31:0/2067819422 conn(0x55a084047de0 msgr2=0x55a084046ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:57.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f9639078740 1 -- 172.21.15.31:0/2067819422 shutdown_connections 2024-09-16T00:46:57.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.383+0000 7f9639078740 1 -- 172.21.15.31:0/2067819422 wait complete. 2024-09-16T00:46:57.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9639078740 1 -- start start 2024-09-16T00:46:57.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9639078740 1 Processor -- start 2024-09-16T00:46:57.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9639078740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08405f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9639078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 0x55a08405f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9639078740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a0840d4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9639078740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96301ab830 con 0x55a08405ae40 2024-09-16T00:46:57.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9639078740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f96301acca0 con 0x55a08404b730 2024-09-16T00:46:57.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9639078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f96301abf00 con 0x55a08404c050 2024-09-16T00:46:57.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9639078740 10 client.0 _ll_register_callbacks cb 0x7f9624004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:46:57.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9634a3b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08405f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9634a3b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08405f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:41228/0 (socket says 172.21.15.31:41228) 2024-09-16T00:46:57.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f9634a3b640 1 -- 172.21.15.31:0/2570103861 learned_addr learned my addr 172.21.15.31:0/2570103861 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:46:57.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.384+0000 7f963523c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a0840d4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f962ffff640 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 0x55a08405f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f9634a3b640 1 -- 172.21.15.31:0/2570103861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 msgr2=0x55a08405f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:57.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f9634a3b640 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 0x55a08405f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f9634a3b640 1 -- 172.21.15.31:0/2570103861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 msgr2=0x55a0840d4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:46:57.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f9634a3b640 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a0840d4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:57.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f9634a3b640 1 -- 172.21.15.31:0/2570103861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0840d4e60 con 0x55a08404b730 2024-09-16T00:46:57.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f963523c640 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a0840d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:46:57.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f9634a3b640 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08405f220 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f962401d5b0 tx=0x7f9624004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127d655f031f3f14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:57.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f960ffff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:46:57.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f960ffff640 1 -- 172.21.15.31:0/2570103861 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624039060 con 0x55a08404b730 2024-09-16T00:46:57.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f960ffff640 1 -- 172.21.15.31:0/2570103861 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f962400f850 con 0x55a08404b730 2024-09-16T00:46:57.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f9639078740 1 -- 172.21.15.31:0/2570103861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961c0c4dc0 con 0x55a08404b730 2024-09-16T00:46:57.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f960ffff640 1 -- 172.21.15.31:0/2570103861 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f962400fa50 con 0x55a08404b730 2024-09-16T00:46:57.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f9639078740 10 client.25727 fetch_fsmap learned FSMap version 20 2024-09-16T00:46:57.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.385+0000 7f9639078740 1 -- 172.21.15.31:0/2570103861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a0840d60b0 con 0x55a08404b730 2024-09-16T00:46:57.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f960ffff640 1 -- 172.21.15.31:0/2570103861 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961c0c4dc0 con 0x55a08404b730 2024-09-16T00:46:57.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f960ffff640 1 -- 172.21.15.31:0/2570103861 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f962401a070 con 0x55a08404b730 2024-09-16T00:46:57.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f9639078740 10 client.25727 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:46:57.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f9639078740 10 client.25727 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:46:57.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f9639078740 20 client.25727 populate_metadata read hostname 'smithi031' 2024-09-16T00:46:57.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f9639078740 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a0840d6840 0x55a0840d8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:57.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f9639078740 4 client.25727 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-16T00:46:57.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f9639078740 1 -- 172.21.15.31:0/2570103861 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x55a083f33f40 con 0x55a0840d6840 2024-09-16T00:46:57.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f962ffff640 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a0840d6840 0x55a0840d8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:57.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f962ffff640 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a0840d6840 0x55a0840d8c60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f014b039fc31c9d server_cookie=e9a2c6d054cda9e9 in_seq=0 out_seq=0 2024-09-16T00:46:57.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:46:57.386+0000 7f960ffff640 10 client.25727 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:46:59.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:58 smithi081 ceph-mon[35539]: pgmap v531: 129 pgs: 129 active+clean; 610 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 972 KiB/s wr, 762 op/s 2024-09-16T00:46:59.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:58 smithi102 ceph-mon[35364]: pgmap v531: 129 pgs: 129 active+clean; 610 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 972 KiB/s wr, 762 op/s 2024-09-16T00:46:59.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:58 smithi031 ceph-mon[30956]: pgmap v531: 129 pgs: 129 active+clean; 610 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 972 KiB/s wr, 762 op/s 2024-09-16T00:47:00.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:47:00.202 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:46:59 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:47:00.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:46:59 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:47:00.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:46:59 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:47:00.457 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.991+0000 7f3254710640 1 -- 172.21.15.31:0/3695461762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f324c101dd0 msgr2=0x7f324c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.991+0000 7f3254710640 1 --2- 172.21.15.31:0/3695461762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f324c101dd0 0x7f324c1021b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3248009900 tx=0x7f324802a410 comp rx=0 tx=0).stop 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.991+0000 7f3254710640 1 -- 172.21.15.31:0/3695461762 shutdown_connections 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.991+0000 7f3254710640 1 --2- 172.21.15.31:0/3695461762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f324c106f00 0x7f324c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.991+0000 7f3254710640 1 --2- 172.21.15.31:0/3695461762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f324c102780 0x7f324c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.991+0000 7f3254710640 1 --2- 172.21.15.31:0/3695461762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f324c101dd0 0x7f324c1021b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.991+0000 7f3254710640 1 -- 172.21.15.31:0/3695461762 >> 172.21.15.31:0/3695461762 conn(0x7f324c0fd890 msgr2=0x7f324c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.991+0000 7f3254710640 1 -- 172.21.15.31:0/3695461762 shutdown_connections 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.991+0000 7f3254710640 1 -- 172.21.15.31:0/3695461762 wait complete. 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.992+0000 7f3254710640 1 Processor -- start 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.992+0000 7f3254710640 1 -- start start 2024-09-16T00:47:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.992+0000 7f3254710640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f324c101dd0 0x7f324c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.992+0000 7f3254710640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f324c102780 0x7f324c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.992+0000 7f3254710640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f324c106f00 0x7f324c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.992+0000 7f3254710640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f324c111f60 con 0x7f324c102780 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.992+0000 7f3254710640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f324c111de0 con 0x7f324c106f00 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.992+0000 7f3254710640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1120e0 con 0x7f324c101dd0 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.992+0000 7f3251cab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f324c102780 0x7f324c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3252cad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f324c106f00 0x7f324c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3252cad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f324c106f00 0x7f324c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:41238/0 (socket says 172.21.15.31:41238) 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3252cad640 1 -- 172.21.15.31:0/2771526552 learned_addr learned my addr 172.21.15.31:0/2771526552 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3251cab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f324c102780 0x7f324c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52252/0 (socket says 172.21.15.31:52252) 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f32524ac640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f324c101dd0 0x7f324c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3252cad640 1 -- 172.21.15.31:0/2771526552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f324c101dd0 msgr2=0x7f324c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3252cad640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f324c101dd0 0x7f324c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3252cad640 1 -- 172.21.15.31:0/2771526552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f324c102780 msgr2=0x7f324c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3252cad640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f324c102780 0x7f324c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3252cad640 1 -- 172.21.15.31:0/2771526552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c19d3e0 con 0x7f324c106f00 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3252cad640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f324c106f00 0x7f324c19cd00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f324400eb10 tx=0x7f324400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f32437fe640 1 -- 172.21.15.31:0/2771526552 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244004b20 con 0x7f324c106f00 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3254710640 1 -- 172.21.15.31:0/2771526552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c19d6d0 con 0x7f324c106f00 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f32437fe640 1 -- 172.21.15.31:0/2771526552 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3244004cc0 con 0x7f324c106f00 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f32437fe640 1 -- 172.21.15.31:0/2771526552 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244010760 con 0x7f324c106f00 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.993+0000 7f3254710640 1 -- 172.21.15.31:0/2771526552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c1a4fb0 con 0x7f324c106f00 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.996+0000 7f32437fe640 1 -- 172.21.15.31:0/2771526552 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3244005020 con 0x7f324c106f00 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.996+0000 7f3256ae9740 1 Processor -- start 2024-09-16T00:47:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.996+0000 7f3256ae9740 1 -- start start 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.996+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 0x55d8c757e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.996+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757edc0 0x55d8c7589140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.996+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 0x55d8c758ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.996+0000 7f3256ae9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d8c74e42e0 con 0x55d8c7589680 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.996+0000 7f3256ae9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d8c748bce0 con 0x55d8c757edc0 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.996+0000 7f3256ae9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d8c74e02a0 con 0x55d8c757e4a0 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32524ac640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757edc0 0x55d8c7589140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f3251cab640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 0x55d8c757e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32524ac640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757edc0 0x55d8c7589140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:41250/0 (socket says 172.21.15.31:41250) 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f3251cab640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 0x55d8c757e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:44542/0 (socket says 172.21.15.31:44542) 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32524ac640 1 -- 172.21.15.31:0/3701228538 learned_addr learned my addr 172.21.15.31:0/3701228538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32437fe640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f32380a3240 0x7f32380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f3251cab640 1 --2- 172.21.15.31:0/3701228538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 0x55d8c758ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f3252cad640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f32380a3240 0x7f32380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32524ac640 1 -- 172.21.15.31:0/3701228538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 msgr2=0x55d8c757e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32524ac640 1 --2- 172.21.15.31:0/3701228538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 0x55d8c757e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32524ac640 1 -- 172.21.15.31:0/3701228538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 msgr2=0x55d8c758ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32437fe640 1 -- 172.21.15.31:0/2771526552 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f32440d01b0 con 0x7f324c106f00 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32524ac640 1 --2- 172.21.15.31:0/3701228538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 0x55d8c758ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32524ac640 1 -- 172.21.15.31:0/3701228538 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c7470440 con 0x55d8c757edc0 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f32524ac640 1 --2- 172.21.15.31:0/3701228538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757edc0 0x55d8c7589140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3248009900 tx=0x7f3248005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f14d59ffd56dcda7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.997+0000 7f3252cad640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f32380a3240 0x7f32380a5700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3244098df0 tx=0x7f3244005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.998+0000 7f3240ff9640 1 -- 172.21.15.31:0/3701228538 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248040070 con 0x55d8c757edc0 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.998+0000 7f3240ff9640 1 -- 172.21.15.31:0/3701228538 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3248031870 con 0x55d8c757edc0 2024-09-16T00:47:01.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.998+0000 7f3240ff9640 1 -- 172.21.15.31:0/3701228538 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248035040 con 0x55d8c757edc0 2024-09-16T00:47:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 -- 172.21.15.31:0/3701228538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757edc0 msgr2=0x55d8c7589140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 --2- 172.21.15.31:0/3701228538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757edc0 0x55d8c7589140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3248009900 tx=0x7f3248005be0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 -- 172.21.15.31:0/3701228538 shutdown_connections 2024-09-16T00:47:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 --2- 172.21.15.31:0/3701228538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 0x55d8c758ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 --2- 172.21.15.31:0/3701228538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757edc0 0x55d8c7589140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 --2- 172.21.15.31:0/3701228538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 0x55d8c757e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 -- 172.21.15.31:0/3701228538 >> 172.21.15.31:0/3701228538 conn(0x55d8c757aa30 msgr2=0x55d8c757ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 -- 172.21.15.31:0/3701228538 shutdown_connections 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 -- 172.21.15.31:0/3701228538 wait complete. 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:00.999+0000 7f3256ae9740 1 -- start start 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3256ae9740 1 Processor -- start 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 0x55d8c7588f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75861f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c7589680 0x55d8c7586730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3256ae9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d8c758c250 con 0x55d8c757edc0 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3256ae9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d8c749f050 con 0x55d8c7589680 2024-09-16T00:47:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3256ae9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d8c758c3d0 con 0x55d8c757e4a0 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3256ae9740 10 client.0 _ll_register_callbacks cb 0x55d8c74df160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f32524ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75861f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f32524ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75861f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52268/0 (socket says 172.21.15.31:52268) 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f32524ac640 1 -- 172.21.15.31:0/334594088 learned_addr learned my addr 172.21.15.31:0/334594088 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3251cab640 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c7589680 0x55d8c7586730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f3251cab640 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 0x55d8c7588f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f32524ac640 1 -- 172.21.15.31:0/334594088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 msgr2=0x55d8c7588f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f32524ac640 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 0x55d8c7588f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f32524ac640 1 -- 172.21.15.31:0/334594088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c7589680 msgr2=0x55d8c7586730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f32524ac640 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c7589680 0x55d8c7586730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.000+0000 7f32524ac640 1 -- 172.21.15.31:0/334594088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c72553c0 con 0x55d8c757edc0 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.001+0000 7f32524ac640 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75861f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3248006010 tx=0x7f324802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ddb0e2719cc1d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.001+0000 7f3235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.001+0000 7f3235ffb640 1 -- 172.21.15.31:0/334594088 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248040030 con 0x55d8c757edc0 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.001+0000 7f3235ffb640 1 -- 172.21.15.31:0/334594088 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f3248004050 con 0x55d8c757edc0 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.001+0000 7f3256ae9740 1 -- 172.21.15.31:0/334594088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3248031870 con 0x55d8c757edc0 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.001+0000 7f3235ffb640 1 -- 172.21.15.31:0/334594088 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3248056050 con 0x55d8c757edc0 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.001+0000 7f3256ae9740 10 client.16068 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.001+0000 7f3256ae9740 1 -- 172.21.15.31:0/334594088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d8c7597bd0 con 0x55d8c757edc0 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.001+0000 7f3235ffb640 1 -- 172.21.15.31:0/334594088 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3248031870 con 0x55d8c757edc0 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3235ffb640 1 -- 172.21.15.31:0/334594088 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f32480035f0 con 0x55d8c757edc0 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3256ae9740 10 client.16068 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3256ae9740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3256ae9740 20 client.16068 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3256ae9740 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8c7598f20 0x55d8c759b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3256ae9740 4 client.16068 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:47:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3256ae9740 1 -- 172.21.15.31:0/334594088 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8c74f1850 con 0x55d8c7598f20 2024-09-16T00:47:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3251cab640 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8c7598f20 0x55d8c759b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3251cab640 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8c7598f20 0x55d8c759b310 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54ce316c07130cf server_cookie=4c64a73d16ade397 in_seq=0 out_seq=0 2024-09-16T00:47:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.002+0000 7f3235ffb640 10 client.16068 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3235ffb640 1 -- 172.21.15.31:0/334594088 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8c74f1850 con 0x55d8c7598f20 2024-09-16T00:47:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3235ffb640 10 client.16068 handle_command_reply: tid=0 2024-09-16T00:47:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 client.16068 shutdown 2024-09-16T00:47:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 -- 172.21.15.31:0/334594088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 msgr2=0x55d8c75861f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75861f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3248006010 tx=0x7f324802a410 comp rx=0 tx=0).stop 2024-09-16T00:47:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 -- 172.21.15.31:0/334594088 shutdown_connections 2024-09-16T00:47:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c7589680 0x55d8c7586730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75861f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8c7598f20 0x55d8c759b310 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 --2- 172.21.15.31:0/334594088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757e4a0 0x55d8c7588f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 -- 172.21.15.31:0/334594088 >> 172.21.15.31:0/334594088 conn(0x55d8c757aa30 msgr2=0x55d8c7580710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3235ffb640 0 client.16068 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 -- 172.21.15.31:0/334594088 shutdown_connections 2024-09-16T00:47:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 1 -- 172.21.15.31:0/334594088 wait complete. 2024-09-16T00:47:01.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.052+0000 7f3256ae9740 20 client.16068 trim_cache size 0 max 16384 2024-09-16T00:47:01.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f3256ae9740 1 Processor -- start 2024-09-16T00:47:01.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f3256ae9740 1 -- start start 2024-09-16T00:47:01.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 0x55d8c7588160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c7589680 0x55d8c75a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f3256ae9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1acca0 con 0x55d8c757edc0 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f3256ae9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1af520 con 0x55d8c757e4a0 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f3256ae9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1ad310 con 0x55d8c7589680 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f32524ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f32524ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52270/0 (socket says 172.21.15.31:52270) 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f32524ac640 1 -- 172.21.15.31:0/1831094384 learned_addr learned my addr 172.21.15.31:0/1831094384 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f3251cab640 1 --2- 172.21.15.31:0/1831094384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 0x55d8c7588160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f32524ac640 1 -- 172.21.15.31:0/1831094384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c7589680 msgr2=0x55d8c75a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f32524ac640 1 --2- 172.21.15.31:0/1831094384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c7589680 0x55d8c75a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f32524ac640 1 -- 172.21.15.31:0/1831094384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 msgr2=0x55d8c7588160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f32524ac640 1 --2- 172.21.15.31:0/1831094384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 0x55d8c7588160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.070+0000 7f32524ac640 1 -- 172.21.15.31:0/1831094384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c75851a0 con 0x55d8c757edc0 2024-09-16T00:47:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.071+0000 7f32524ac640 1 --2- 172.21.15.31:0/1831094384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75a08b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3248005ec0 tx=0x7f3248005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c844e6c9282fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.071+0000 7f3237fff640 1 -- 172.21.15.31:0/1831094384 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324805f070 con 0x55d8c757edc0 2024-09-16T00:47:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.071+0000 7f3237fff640 1 -- 172.21.15.31:0/1831094384 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f324804d070 con 0x55d8c757edc0 2024-09-16T00:47:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.071+0000 7f3237fff640 1 -- 172.21.15.31:0/1831094384 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248040070 con 0x55d8c757edc0 2024-09-16T00:47:01.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.071+0000 7f3256ae9740 1 -- 172.21.15.31:0/1831094384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 msgr2=0x55d8c75a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.071+0000 7f3256ae9740 1 --2- 172.21.15.31:0/1831094384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75a08b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3248005ec0 tx=0x7f3248005900 comp rx=0 tx=0).stop 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 -- 172.21.15.31:0/1831094384 shutdown_connections 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 --2- 172.21.15.31:0/1831094384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c7589680 0x55d8c75a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 --2- 172.21.15.31:0/1831094384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c757edc0 0x55d8c75a08b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 --2- 172.21.15.31:0/1831094384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 0x55d8c7588160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 -- 172.21.15.31:0/1831094384 >> 172.21.15.31:0/1831094384 conn(0x55d8c757aa30 msgr2=0x55d8c7587a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 -- 172.21.15.31:0/1831094384 shutdown_connections 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 -- 172.21.15.31:0/1831094384 wait complete. 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 -- start start 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 Processor -- start 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 0x55d8c758f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757edc0 0x55d8c758f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 0x55d8c758fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1ad9e0 con 0x55d8c7589680 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1aee50 con 0x55d8c757e4a0 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1ae0b0 con 0x55d8c757edc0 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.072+0000 7f3256ae9740 10 client.0 _ll_register_callbacks cb 0x7f323c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f3251cab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 0x55d8c758f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f3251cab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 0x55d8c758f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:41288/0 (socket says 172.21.15.31:41288) 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f3251cab640 1 -- 172.21.15.31:0/983985524 learned_addr learned my addr 172.21.15.31:0/983985524 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f32524ac640 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757edc0 0x55d8c758f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f3251cab640 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 0x55d8c758fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f32524ac640 1 -- 172.21.15.31:0/983985524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 msgr2=0x55d8c758f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f32524ac640 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 0x55d8c758f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f32524ac640 1 -- 172.21.15.31:0/983985524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 msgr2=0x55d8c758fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f32524ac640 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 0x55d8c758fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f32524ac640 1 -- 172.21.15.31:0/983985524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c757ebf0 con 0x55d8c757edc0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f32524ac640 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757edc0 0x55d8c758f910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55d8c7579220 tx=0x7f3248003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c943eac96eca59af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f3235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f3235ffb640 1 -- 172.21.15.31:0/983985524 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324805f070 con 0x55d8c757edc0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f3235ffb640 1 -- 172.21.15.31:0/983985524 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f3248003b60 con 0x55d8c757edc0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.073+0000 7f3256ae9740 1 -- 172.21.15.31:0/983985524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f324804d070 con 0x55d8c757edc0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3235ffb640 1 -- 172.21.15.31:0/983985524 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3248040030 con 0x55d8c757edc0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3256ae9740 10 client.25672 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3256ae9740 1 -- 172.21.15.31:0/983985524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d8c76039b0 con 0x55d8c757edc0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3235ffb640 1 -- 172.21.15.31:0/983985524 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f324804d070 con 0x55d8c757edc0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3235ffb640 1 -- 172.21.15.31:0/983985524 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f3248004880 con 0x55d8c757edc0 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3256ae9740 10 client.25672 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3256ae9740 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3256ae9740 20 client.25672 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3256ae9740 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8c7603ff0 0x55d8c7606410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3256ae9740 4 client.25672 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:47:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3256ae9740 1 -- 172.21.15.31:0/983985524 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8c74ebd60 con 0x55d8c7603ff0 2024-09-16T00:47:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.074+0000 7f3251cab640 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8c7603ff0 0x55d8c7606410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.075+0000 7f3235ffb640 10 client.25672 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.075+0000 7f3251cab640 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8c7603ff0 0x55d8c7606410 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac5b3ca6827fbbe server_cookie=97234e62c1547d3 in_seq=0 out_seq=0 2024-09-16T00:47:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3235ffb640 1 -- 172.21.15.31:0/983985524 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d8c74ebd60 con 0x55d8c7603ff0 2024-09-16T00:47:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3235ffb640 10 client.25672 handle_command_reply: tid=0 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3256ae9740 1 client.25672 shutdown 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3256ae9740 1 -- 172.21.15.31:0/983985524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757edc0 msgr2=0x55d8c758f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3256ae9740 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757edc0 0x55d8c758f910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55d8c7579220 tx=0x7f3248003820 comp rx=0 tx=0).stop 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3256ae9740 1 -- 172.21.15.31:0/983985524 shutdown_connections 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3256ae9740 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8c7589680 0x55d8c758fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3256ae9740 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d8c7603ff0 0x55d8c7606410 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3256ae9740 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d8c757edc0 0x55d8c758f910 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3256ae9740 1 --2- 172.21.15.31:0/983985524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d8c757e4a0 0x55d8c758f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3235ffb640 0 client.25672 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.145+0000 7f3256ae9740 1 -- 172.21.15.31:0/983985524 >> 172.21.15.31:0/983985524 conn(0x55d8c757aa30 msgr2=0x55d8c7579350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3256ae9740 1 -- 172.21.15.31:0/983985524 shutdown_connections 2024-09-16T00:47:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3256ae9740 1 -- 172.21.15.31:0/983985524 wait complete. 2024-09-16T00:47:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3256ae9740 20 client.25672 trim_cache size 0 max 16384 2024-09-16T00:47:01.149 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:47:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:47:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:47:01.149 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:47:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 -- 172.21.15.31:0/2771526552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f32380a3240 msgr2=0x7f32380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f32380a3240 0x7f32380a5700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3244098df0 tx=0x7f3244005b40 comp rx=0 tx=0).stop 2024-09-16T00:47:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 -- 172.21.15.31:0/2771526552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f324c106f00 msgr2=0x7f324c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f324c106f00 0x7f324c19cd00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f324400eb10 tx=0x7f324400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 -- 172.21.15.31:0/2771526552 shutdown_connections 2024-09-16T00:47:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f324c106f00 0x7f324c19cd00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f324c102780 0x7f324c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f32380a3240 0x7f32380a5700 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 --2- 172.21.15.31:0/2771526552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f324c101dd0 0x7f324c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 -- 172.21.15.31:0/2771526552 >> 172.21.15.31:0/2771526552 conn(0x7f324c0fd890 msgr2=0x7f324c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.146+0000 7f3254710640 1 -- 172.21.15.31:0/2771526552 shutdown_connections 2024-09-16T00:47:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:01.147+0000 7f3254710640 1 -- 172.21.15.31:0/2771526552 wait complete. 2024-09-16T00:47:01.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:00 smithi031 ceph-mon[30956]: pgmap v532: 129 pgs: 129 active+clean; 612 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 990 KiB/s wr, 691 op/s 2024-09-16T00:47:01.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:00 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:47:01.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:00 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:47:01.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:00 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:47:01.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:00 smithi081 ceph-mon[35539]: pgmap v532: 129 pgs: 129 active+clean; 612 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 990 KiB/s wr, 691 op/s 2024-09-16T00:47:01.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:00 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:47:01.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:00 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:47:01.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:00 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:47:01.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:00 smithi102 ceph-mon[35364]: pgmap v532: 129 pgs: 129 active+clean; 612 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 990 KiB/s wr, 691 op/s 2024-09-16T00:47:01.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:00 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:47:01.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:00 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:47:01.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:00 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:47:02.050 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:47:02.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:47:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:02.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.842+0000 7fe9ab51f640 1 -- 172.21.15.31:0/1612009280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4102d90 msgr2=0x7fe9a4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:02.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.842+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/1612009280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4102d90 0x7fe9a4103170 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009930 tx=0x7fe99c02a420 comp rx=0 tx=0).stop 2024-09-16T00:47:02.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.842+0000 7fe9ab51f640 1 -- 172.21.15.31:0/1612009280 shutdown_connections 2024-09-16T00:47:02.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.842+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/1612009280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a4108290 0x7fe9a410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.842+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/1612009280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9a4103740 0x7fe9a4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.842+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/1612009280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4102d90 0x7fe9a4103170 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.842+0000 7fe9ab51f640 1 -- 172.21.15.31:0/1612009280 >> 172.21.15.31:0/1612009280 conn(0x7fe9a40fc7d0 msgr2=0x7fe9a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.842+0000 7fe9ab51f640 1 -- 172.21.15.31:0/1612009280 shutdown_connections 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.842+0000 7fe9ab51f640 1 -- 172.21.15.31:0/1612009280 wait complete. 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9ab51f640 1 Processor -- start 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9ab51f640 1 -- start start 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9ab51f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9a4102d90 0x7fe9a41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9ab51f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4103740 0x7fe9a4100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:02.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9ab51f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a4108290 0x7fe9a4101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9ab51f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41120c0 con 0x7fe9a4103740 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9ab51f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4111f40 con 0x7fe9a4102d90 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9ab51f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4112240 con 0x7fe9a4108290 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a8aba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4103740 0x7fe9a4100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a92bb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9a4102d90 0x7fe9a41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a8aba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4103740 0x7fe9a4100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52308/0 (socket says 172.21.15.31:52308) 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a8aba640 1 -- 172.21.15.31:0/2376290041 learned_addr learned my addr 172.21.15.31:0/2376290041 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a9abc640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a4108290 0x7fe9a4101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a8aba640 1 -- 172.21.15.31:0/2376290041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a4108290 msgr2=0x7fe9a4101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a8aba640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a4108290 0x7fe9a4101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:02.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a8aba640 1 -- 172.21.15.31:0/2376290041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9a4102d90 msgr2=0x7fe9a41009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:02.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a8aba640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9a4102d90 0x7fe9a41009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:02.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a8aba640 1 -- 172.21.15.31:0/2376290041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a41a6a50 con 0x7fe9a4103740 2024-09-16T00:47:02.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.843+0000 7fe9a92bb640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9a4102d90 0x7fe9a41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:02.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.844+0000 7fe9a8aba640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4103740 0x7fe9a4100f30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe994005fd0 tx=0x7fe99400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:02.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.844+0000 7fe99a7fc640 1 -- 172.21.15.31:0/2376290041 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994017070 con 0x7fe9a4103740 2024-09-16T00:47:02.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.844+0000 7fe99a7fc640 1 -- 172.21.15.31:0/2376290041 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994009c30 con 0x7fe9a4103740 2024-09-16T00:47:02.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.844+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a41a6d40 con 0x7fe9a4103740 2024-09-16T00:47:02.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.844+0000 7fe99a7fc640 1 -- 172.21.15.31:0/2376290041 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994010460 con 0x7fe9a4103740 2024-09-16T00:47:02.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.844+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a41a7270 con 0x7fe9a4103740 2024-09-16T00:47:02.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.845+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a406a7f0 con 0x7fe9a4103740 2024-09-16T00:47:02.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.847+0000 7fe99a7fc640 1 -- 172.21.15.31:0/2376290041 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe994003e70 con 0x7fe9a4103740 2024-09-16T00:47:02.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.848+0000 7fe99a7fc640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9900a32c0 0x7fe9900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:02.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.848+0000 7fe9a92bb640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9900a32c0 0x7fe9900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:02.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.848+0000 7fe99a7fc640 1 -- 172.21.15.31:0/2376290041 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fe9940cffc0 con 0x7fe9a4103740 2024-09-16T00:47:02.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.848+0000 7fe9a92bb640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9900a32c0 0x7fe9900a5780 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe99c006010 tx=0x7fe99c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:02.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:02.852+0000 7fe99a7fc640 1 -- 172.21.15.31:0/2376290041 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe994020030 con 0x7fe9a4103740 2024-09-16T00:47:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.079+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9a4103170 con 0x7fe9a4103740 2024-09-16T00:47:03.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:02 smithi031 ceph-mon[30956]: pgmap v533: 129 pgs: 129 active+clean; 611 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 607 KiB/s wr, 558 op/s 2024-09-16T00:47:03.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.080+0000 7fe99a7fc640 1 -- 172.21.15.31:0/2376290041 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fe9a4103170 con 0x7fe9a4103740 2024-09-16T00:47:03.083 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:47:03.083 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:47:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:47:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9900a32c0 msgr2=0x7fe9900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9900a32c0 0x7fe9900a5780 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe99c006010 tx=0x7fe99c031510 comp rx=0 tx=0).stop 2024-09-16T00:47:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4103740 msgr2=0x7fe9a4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4103740 0x7fe9a4100f30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe994005fd0 tx=0x7fe99400dde0 comp rx=0 tx=0).stop 2024-09-16T00:47:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 shutdown_connections 2024-09-16T00:47:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe9900a32c0 0x7fe9900a5780 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a4108290 0x7fe9a4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:03.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4103740 0x7fe9a4100f30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 --2- 172.21.15.31:0/2376290041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9a4102d90 0x7fe9a41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 >> 172.21.15.31:0/2376290041 conn(0x7fe9a40fc7d0 msgr2=0x7fe9a4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 shutdown_connections 2024-09-16T00:47:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:03.082+0000 7fe9ab51f640 1 -- 172.21.15.31:0/2376290041 wait complete. 2024-09-16T00:47:03.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:02 smithi081 ceph-mon[35539]: pgmap v533: 129 pgs: 129 active+clean; 611 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 607 KiB/s wr, 558 op/s 2024-09-16T00:47:03.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:02 smithi102 ceph-mon[35364]: pgmap v533: 129 pgs: 129 active+clean; 611 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 607 KiB/s wr, 558 op/s 2024-09-16T00:47:04.022 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:47:04.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:03 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2376290041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:47:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:04.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:03 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2376290041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:47:04.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:03 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2376290041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:47:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 -- 172.21.15.31:0/91759804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4106f10 msgr2=0x7f9ac410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 --2- 172.21.15.31:0/91759804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4106f10 0x7f9ac410ea90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009ca0 tx=0x7f9abc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 -- 172.21.15.31:0/91759804 shutdown_connections 2024-09-16T00:47:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 --2- 172.21.15.31:0/91759804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4106f10 0x7f9ac410ea90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 --2- 172.21.15.31:0/91759804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ac4102790 0x7f9ac41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 --2- 172.21.15.31:0/91759804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ac4101de0 0x7f9ac41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 -- 172.21.15.31:0/91759804 >> 172.21.15.31:0/91759804 conn(0x7f9ac40fd8c0 msgr2=0x7f9ac40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 -- 172.21.15.31:0/91759804 shutdown_connections 2024-09-16T00:47:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 -- 172.21.15.31:0/91759804 wait complete. 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 Processor -- start 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.822+0000 7f9acb130640 1 -- start start 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9acb130640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ac4101de0 0x7f9ac40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9acb130640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4102790 0x7f9ac4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9acb130640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ac4106f10 0x7f9ac4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9acb130640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac4111f70 con 0x7f9ac4102790 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9acb130640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac4111df0 con 0x7f9ac4106f10 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9acb130640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41120f0 con 0x7f9ac4101de0 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4102790 0x7f9ac4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4102790 0x7f9ac4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52324/0 (socket says 172.21.15.31:52324) 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac3fff640 1 -- 172.21.15.31:0/3099042055 learned_addr learned my addr 172.21.15.31:0/3099042055 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac8ecc640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ac4101de0 0x7f9ac40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac96cd640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ac4106f10 0x7f9ac4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac3fff640 1 -- 172.21.15.31:0/3099042055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ac4101de0 msgr2=0x7f9ac40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ac4101de0 0x7f9ac40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac3fff640 1 -- 172.21.15.31:0/3099042055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ac4106f10 msgr2=0x7f9ac4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ac4106f10 0x7f9ac4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac3fff640 1 -- 172.21.15.31:0/3099042055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac41a6870 con 0x7f9ac4102790 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4102790 0x7f9ac4074a30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e9f0 tx=0x7f9ab000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9ac1ffb640 1 -- 172.21.15.31:0/3099042055 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000cd80 con 0x7f9ac4102790 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.823+0000 7f9acb130640 1 -- 172.21.15.31:0/3099042055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac41a6b60 con 0x7f9ac4102790 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.824+0000 7f9acb130640 1 -- 172.21.15.31:0/3099042055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac41a70f0 con 0x7f9ac4102790 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.824+0000 7f9ac1ffb640 1 -- 172.21.15.31:0/3099042055 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab0010910 con 0x7f9ac4102790 2024-09-16T00:47:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.824+0000 7f9ac1ffb640 1 -- 172.21.15.31:0/3099042055 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0018980 con 0x7f9ac4102790 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.826+0000 7f9ac1ffb640 1 -- 172.21.15.31:0/3099042055 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9ab0010ac0 con 0x7f9ac4102790 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.826+0000 7f9acd509740 1 Processor -- start 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.826+0000 7f9acd509740 1 -- start start 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9acd509740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f21b40 0x55d4e9f20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9acd509740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f20a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac1ffb640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9ab40a3240 0x7f9ab40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9acd509740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9acd509740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9e862f0 con 0x55d4e9f21b40 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9acd509740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9e2dcf0 con 0x55d4e9f2b370 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9acd509740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9e822b0 con 0x55d4e9f205d0 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac96cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f21b40 0x55d4e9f20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac96cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f21b40 0x55d4e9f20090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52338/0 (socket says 172.21.15.31:52338) 2024-09-16T00:47:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac96cd640 1 -- 172.21.15.31:0/1811812833 learned_addr learned my addr 172.21.15.31:0/1811812833 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac8ecc640 1 --2- 172.21.15.31:0/1811812833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f20a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/1811812833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac96cd640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9ab40a3240 0x7f9ab40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac1ffb640 1 -- 172.21.15.31:0/3099042055 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f9ab00cfea0 con 0x7f9ac4102790 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac3fff640 1 -- 172.21.15.31:0/1811812833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 msgr2=0x55d4e9f20a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/1811812833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f20a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac3fff640 1 -- 172.21.15.31:0/1811812833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f21b40 msgr2=0x55d4e9f20090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/1811812833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f21b40 0x55d4e9f20090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac3fff640 1 -- 172.21.15.31:0/1811812833 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e9e123a0 con 0x55d4e9f2b370 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/1811812833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2d760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0098620 tx=0x7f9ab0025800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ec7c4ccdf863fc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9ac96cd640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9ab40a3240 0x7f9ab40a5700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009a60 tx=0x7f9abc004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.827+0000 7f9aaeffd640 1 -- 172.21.15.31:0/1811812833 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab009bb20 con 0x55d4e9f2b370 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9aaeffd640 1 -- 172.21.15.31:0/1811812833 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab0010db0 con 0x55d4e9f2b370 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9aaeffd640 1 -- 172.21.15.31:0/1811812833 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00aec50 con 0x55d4e9f2b370 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9acd509740 1 -- 172.21.15.31:0/1811812833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 msgr2=0x55d4e9f2d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9acd509740 1 --2- 172.21.15.31:0/1811812833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2d760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0098620 tx=0x7f9ab0025800 comp rx=0 tx=0).stop 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9acd509740 1 -- 172.21.15.31:0/1811812833 shutdown_connections 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9acd509740 1 --2- 172.21.15.31:0/1811812833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2d760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9acd509740 1 --2- 172.21.15.31:0/1811812833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f20a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9acd509740 1 --2- 172.21.15.31:0/1811812833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f21b40 0x55d4e9f20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9acd509740 1 -- 172.21.15.31:0/1811812833 >> 172.21.15.31:0/1811812833 conn(0x55d4e9ee9310 msgr2=0x55d4e9f1cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9acd509740 1 -- 172.21.15.31:0/1811812833 shutdown_connections 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.828+0000 7f9acd509740 1 -- 172.21.15.31:0/1811812833 wait complete. 2024-09-16T00:47:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.829+0000 7f9acd509740 1 -- start start 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.829+0000 7f9acd509740 1 Processor -- start 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.829+0000 7f9acd509740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f2ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.829+0000 7f9acd509740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f28010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.829+0000 7f9acd509740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f28600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.829+0000 7f9acd509740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9f2e070 con 0x55d4e9f2b370 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.829+0000 7f9acd509740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9e41060 con 0x55d4e9f21b40 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.829+0000 7f9acd509740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9f2e1f0 con 0x55d4e9f205d0 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.829+0000 7f9acd509740 10 client.0 _ll_register_callbacks cb 0x55d4e9b948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac96cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f28600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f28010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac96cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f28600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52352/0 (socket says 172.21.15.31:52352) 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac96cd640 1 -- 172.21.15.31:0/2414635746 learned_addr learned my addr 172.21.15.31:0/2414635746 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac8ecc640 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f2ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac96cd640 1 -- 172.21.15.31:0/2414635746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 msgr2=0x55d4e9f2ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac96cd640 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f2ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac96cd640 1 -- 172.21.15.31:0/2414635746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 msgr2=0x55d4e9f28010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac96cd640 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f28010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac96cd640 1 -- 172.21.15.31:0/2414635746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e9bf73c0 con 0x55d4e9f2b370 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f28010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9ac96cd640 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f28600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9abc005e30 tx=0x7f9abc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b73a6fdbde2ada5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9aa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2414635746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0309d0 con 0x55d4e9f2b370 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9acd509740 1 -- 172.21.15.31:0/2414635746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab0010db0 con 0x55d4e9f2b370 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2414635746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f9abc030b70 con 0x55d4e9f2b370 2024-09-16T00:47:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2414635746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9abc02f390 con 0x55d4e9f2b370 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9acd509740 10 client.16092 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.830+0000 7f9acd509740 1 -- 172.21.15.31:0/2414635746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d4e9f39b40 con 0x55d4e9f2b370 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.831+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2414635746 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab0010db0 con 0x55d4e9f2b370 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.831+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2414635746 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f9abc048040 con 0x55d4e9f2b370 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.831+0000 7f9acd509740 10 client.16092 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.831+0000 7f9acd509740 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.831+0000 7f9acd509740 20 client.16092 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.831+0000 7f9acd509740 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d4e9f3afa0 0x55d4e9f3d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.831+0000 7f9acd509740 4 client.16092 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.831+0000 7f9acd509740 1 -- 172.21.15.31:0/2414635746 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4e9e93860 con 0x55d4e9f3afa0 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.831+0000 7f9ac8ecc640 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d4e9f3afa0 0x55d4e9f3d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.832+0000 7f9ac8ecc640 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d4e9f3afa0 0x55d4e9f3d3e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bee5eb1755c5cc4c server_cookie=c5426b1202008408 in_seq=0 out_seq=0 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.832+0000 7f9aa3fff640 10 client.16092 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2414635746 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4e9e93860 con 0x55d4e9f3afa0 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9aa3fff640 10 client.16092 handle_command_reply: tid=0 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 client.16092 shutdown 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 -- 172.21.15.31:0/2414635746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 msgr2=0x55d4e9f28600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f28600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9abc005e30 tx=0x7f9abc0095c0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 -- 172.21.15.31:0/2414635746 shutdown_connections 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f28600 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f28010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d4e9f3afa0 0x55d4e9f3d3e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 --2- 172.21.15.31:0/2414635746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f2ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9aa3fff640 0 client.16092 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 -- 172.21.15.31:0/2414635746 >> 172.21.15.31:0/2414635746 conn(0x55d4e9ee9310 msgr2=0x55d4e9f229c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 -- 172.21.15.31:0/2414635746 shutdown_connections 2024-09-16T00:47:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 1 -- 172.21.15.31:0/2414635746 wait complete. 2024-09-16T00:47:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.833+0000 7f9acd509740 20 client.16092 trim_cache size 0 max 16384 2024-09-16T00:47:04.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9acd509740 1 Processor -- start 2024-09-16T00:47:04.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9acd509740 1 -- start start 2024-09-16T00:47:04.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9acd509740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f29f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9acd509740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f23120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9acd509740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f23660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9acd509740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800db80 con 0x55d4e9f2b370 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9acd509740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa8010400 con 0x55d4e9f21b40 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9acd509740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800e1f0 con 0x55d4e9f205d0 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f23120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac96cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f23660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f23120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:41358/0 (socket says 172.21.15.31:41358) 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac3fff640 1 -- 172.21.15.31:0/609716130 learned_addr learned my addr 172.21.15.31:0/609716130 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac8ecc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f29f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac3fff640 1 -- 172.21.15.31:0/609716130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 msgr2=0x55d4e9f29f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:04.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/609716130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f29f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac3fff640 1 -- 172.21.15.31:0/609716130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 msgr2=0x55d4e9f23660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/609716130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f23660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac3fff640 1 -- 172.21.15.31:0/609716130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e9f1ddd0 con 0x55d4e9f21b40 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.848+0000 7f9ac96cd640 1 --2- 172.21.15.31:0/609716130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f23660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.849+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/609716130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f23120 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ab009c150 tx=0x7f9ab0024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc15fb0cbdd558f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.849+0000 7f9aae7fc640 1 -- 172.21.15.31:0/609716130 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00aee30 con 0x55d4e9f21b40 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.849+0000 7f9aae7fc640 1 -- 172.21.15.31:0/609716130 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab0010db0 con 0x55d4e9f21b40 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.849+0000 7f9aae7fc640 1 -- 172.21.15.31:0/609716130 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00af010 con 0x55d4e9f21b40 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.849+0000 7f9acd509740 1 -- 172.21.15.31:0/609716130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 msgr2=0x55d4e9f23120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.849+0000 7f9acd509740 1 --2- 172.21.15.31:0/609716130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f23120 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ab009c150 tx=0x7f9ab0024000 comp rx=0 tx=0).stop 2024-09-16T00:47:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 -- 172.21.15.31:0/609716130 shutdown_connections 2024-09-16T00:47:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 --2- 172.21.15.31:0/609716130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f2b370 0x55d4e9f23660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 --2- 172.21.15.31:0/609716130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f21b40 0x55d4e9f23120 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 --2- 172.21.15.31:0/609716130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f205d0 0x55d4e9f29f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 -- 172.21.15.31:0/609716130 >> 172.21.15.31:0/609716130 conn(0x55d4e9ee9310 msgr2=0x55d4e9f29870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 -- 172.21.15.31:0/609716130 shutdown_connections 2024-09-16T00:47:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 -- 172.21.15.31:0/609716130 wait complete. 2024-09-16T00:47:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 -- start start 2024-09-16T00:47:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 Processor -- start 2024-09-16T00:47:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f205d0 0x55d4e9f33610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f21b40 0x55d4e9f33b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800e8c0 con 0x55d4e9f205d0 2024-09-16T00:47:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800fd30 con 0x55d4e9f2b370 2024-09-16T00:47:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800ef90 con 0x55d4e9f21b40 2024-09-16T00:47:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.850+0000 7f9acd509740 10 client.0 _ll_register_callbacks cb 0x7f9abc0318d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac8ecc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f205d0 0x55d4e9f33610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac8ecc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f205d0 0x55d4e9f33610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52358/0 (socket says 172.21.15.31:52358) 2024-09-16T00:47:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac96cd640 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac8ecc640 1 -- 172.21.15.31:0/2705324636 learned_addr learned my addr 172.21.15.31:0/2705324636 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f21b40 0x55d4e9f33b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac8ecc640 1 -- 172.21.15.31:0/2705324636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f21b40 msgr2=0x55d4e9f33b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac8ecc640 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f21b40 0x55d4e9f33b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac8ecc640 1 -- 172.21.15.31:0/2705324636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 msgr2=0x55d4e9f2e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac8ecc640 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac8ecc640 1 -- 172.21.15.31:0/2705324636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e9f2f200 con 0x55d4e9f205d0 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac96cd640 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9ac8ecc640 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f205d0 0x55d4e9f33610 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8004600 tx=0x7f9ab800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2ea3684d1c227b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9aa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2705324636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8011440 con 0x55d4e9f205d0 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2705324636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f9ab80115e0 con 0x55d4e9f205d0 2024-09-16T00:47:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9acd509740 1 -- 172.21.15.31:0/2705324636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab0010db0 con 0x55d4e9f205d0 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.851+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2705324636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9ab8031050 con 0x55d4e9f205d0 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9acd509740 10 client.16098 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9acd509740 1 -- 172.21.15.31:0/2705324636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d4e9fa5840 con 0x55d4e9f205d0 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2705324636 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab0010db0 con 0x55d4e9f205d0 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2705324636 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f9ab8026970 con 0x55d4e9f205d0 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9acd509740 10 client.16098 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9acd509740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9acd509740 20 client.16098 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9acd509740 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d4e9fa5fb0 0x55d4e9fa83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9acd509740 4 client.16098 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9acd509740 1 -- 172.21.15.31:0/2705324636 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4e9e8dd70 con 0x55d4e9fa5fb0 2024-09-16T00:47:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.852+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d4e9fa5fb0 0x55d4e9fa83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.853+0000 7f9aa3fff640 10 client.16098 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.853+0000 7f9ac3fff640 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d4e9fa5fb0 0x55d4e9fa83f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f300bcad1ec6dc67 server_cookie=57be1a232455195d in_seq=0 out_seq=0 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.853+0000 7f9aa3fff640 1 -- 172.21.15.31:0/2705324636 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4e9e8dd70 con 0x55d4e9fa5fb0 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.853+0000 7f9aa3fff640 10 client.16098 handle_command_reply: tid=0 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.853+0000 7f9acd509740 1 client.16098 shutdown 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.853+0000 7f9acd509740 1 -- 172.21.15.31:0/2705324636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f205d0 msgr2=0x55d4e9f33610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.853+0000 7f9acd509740 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f205d0 0x55d4e9f33610 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8004600 tx=0x7f9ab800be30 comp rx=0 tx=0).stop 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acd509740 1 -- 172.21.15.31:0/2705324636 shutdown_connections 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acd509740 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4e9f2b370 0x55d4e9f2e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acd509740 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55d4e9fa5fb0 0x55d4e9fa83f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9aa3fff640 0 client.16098 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acd509740 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4e9f21b40 0x55d4e9f33b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acd509740 1 --2- 172.21.15.31:0/2705324636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4e9f205d0 0x55d4e9f33610 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acd509740 1 -- 172.21.15.31:0/2705324636 >> 172.21.15.31:0/2705324636 conn(0x55d4e9ee9310 msgr2=0x55d4e9f29870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acd509740 1 -- 172.21.15.31:0/2705324636 shutdown_connections 2024-09-16T00:47:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acd509740 1 -- 172.21.15.31:0/2705324636 wait complete. 2024-09-16T00:47:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acd509740 20 client.16098 trim_cache size 0 max 16384 2024-09-16T00:47:04.864 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:47:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 -- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9ab40a3240 msgr2=0x7f9ab40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9ab40a3240 0x7f9ab40a5700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009a60 tx=0x7f9abc004a80 comp rx=0 tx=0).stop 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 -- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4102790 msgr2=0x7f9ac4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4102790 0x7f9ac4074a30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e9f0 tx=0x7f9ab000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 -- 172.21.15.31:0/3099042055 shutdown_connections 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ac4106f10 0x7f9ac4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac4102790 0x7f9ac4074a30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9ab40a3240 0x7f9ab40a5700 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 --2- 172.21.15.31:0/3099042055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ac4101de0 0x7f9ac40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.854+0000 7f9acb130640 1 -- 172.21.15.31:0/3099042055 >> 172.21.15.31:0/3099042055 conn(0x7f9ac40fd8c0 msgr2=0x7f9ac40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.855+0000 7f9acb130640 1 -- 172.21.15.31:0/3099042055 shutdown_connections 2024-09-16T00:47:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:04.855+0000 7f9acb130640 1 -- 172.21.15.31:0/3099042055 wait complete. 2024-09-16T00:47:05.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:04 smithi031 ceph-mon[30956]: pgmap v534: 129 pgs: 129 active+clean; 613 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 527 KiB/s wr, 431 op/s 2024-09-16T00:47:05.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:04 smithi081 ceph-mon[35539]: pgmap v534: 129 pgs: 129 active+clean; 613 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 527 KiB/s wr, 431 op/s 2024-09-16T00:47:05.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:04 smithi102 ceph-mon[35364]: pgmap v534: 129 pgs: 129 active+clean; 613 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 527 KiB/s wr, 431 op/s 2024-09-16T00:47:06.190 INFO:tasks.workunit.client.0.smithi031.stdout: 117 2298 31 1266 15 1460 65 2024-09-16T00:47:06.683 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:47:06.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:47:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:07.089 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:06 smithi031 ceph-mon[30956]: pgmap v535: 129 pgs: 129 active+clean; 615 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 763 KiB/s wr, 315 op/s 2024-09-16T00:47:07.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:06 smithi081 ceph-mon[35539]: pgmap v535: 129 pgs: 129 active+clean; 615 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 763 KiB/s wr, 315 op/s 2024-09-16T00:47:07.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:06 smithi102 ceph-mon[35364]: pgmap v535: 129 pgs: 129 active+clean; 615 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 763 KiB/s wr, 315 op/s 2024-09-16T00:47:07.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.495+0000 7fa54afef640 1 -- 172.21.15.31:0/2000486552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa544102d30 msgr2=0x7fa544103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:07.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.495+0000 7fa54afef640 1 --2- 172.21.15.31:0/2000486552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa544102d30 0x7fa544103110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa538009930 tx=0x7fa53802a420 comp rx=0 tx=0).stop 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.495+0000 7fa54afef640 1 -- 172.21.15.31:0/2000486552 shutdown_connections 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.495+0000 7fa54afef640 1 --2- 172.21.15.31:0/2000486552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5441082a0 0x7fa54410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.495+0000 7fa54afef640 1 --2- 172.21.15.31:0/2000486552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5441036e0 0x7fa544107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.495+0000 7fa54afef640 1 --2- 172.21.15.31:0/2000486552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa544102d30 0x7fa544103110 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.495+0000 7fa54afef640 1 -- 172.21.15.31:0/2000486552 >> 172.21.15.31:0/2000486552 conn(0x7fa5440fc7d0 msgr2=0x7fa5440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.495+0000 7fa54afef640 1 -- 172.21.15.31:0/2000486552 shutdown_connections 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.495+0000 7fa54afef640 1 -- 172.21.15.31:0/2000486552 wait complete. 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54afef640 1 Processor -- start 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54afef640 1 -- start start 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54afef640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa544102d30 0x7fa5441985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54afef640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5441036e0 0x7fa544198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54afef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5441082a0 0x7fa54419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54afef640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441120a0 con 0x7fa5441082a0 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54afef640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa544111f20 con 0x7fa5441036e0 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54afef640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa544112220 con 0x7fa544102d30 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54958c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5441082a0 0x7fa54419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54958c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5441082a0 0x7fa54419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38084/0 (socket says 172.21.15.31:38084) 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa54958c640 1 -- 172.21.15.31:0/246397701 learned_addr learned my addr 172.21.15.31:0/246397701 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa548d8b640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa544102d30 0x7fa5441985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.496+0000 7fa543fff640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5441036e0 0x7fa544198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa54958c640 1 -- 172.21.15.31:0/246397701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa544102d30 msgr2=0x7fa5441985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa54958c640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa544102d30 0x7fa5441985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa54958c640 1 -- 172.21.15.31:0/246397701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5441036e0 msgr2=0x7fa544198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa54958c640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5441036e0 0x7fa544198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa54958c640 1 -- 172.21.15.31:0/246397701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54419d570 con 0x7fa5441082a0 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa54958c640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5441082a0 0x7fa54419ce90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00eb10 tx=0x7fa53c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa541ffb640 1 -- 172.21.15.31:0/246397701 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c004b20 con 0x7fa5441082a0 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa54afef640 1 -- 172.21.15.31:0/246397701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54419d860 con 0x7fa5441082a0 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa541ffb640 1 -- 172.21.15.31:0/246397701 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c004cc0 con 0x7fa5441082a0 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa541ffb640 1 -- 172.21.15.31:0/246397701 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c010760 con 0x7fa5441082a0 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.497+0000 7fa54afef640 1 -- 172.21.15.31:0/246397701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5441a5140 con 0x7fa5441082a0 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa541ffb640 1 -- 172.21.15.31:0/246397701 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa53c005020 con 0x7fa5441082a0 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa54d3c8740 1 Processor -- start 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa54d3c8740 1 -- start start 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa541ffb640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa5340a3240 0x7fa5340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 0x55ed92b59f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b64800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b64d40 0x55ed92b67130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed92abf280 con 0x55ed92b5a480 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ed92a66b10 con 0x55ed92b64d40 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.500+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed92abb240 con 0x55ed92b59b60 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa543fff640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa5340a3240 0x7fa5340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa541ffb640 1 -- 172.21.15.31:0/246397701 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa53c0d01b0 con 0x7fa5441082a0 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa54958c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b64d40 0x55ed92b67130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa543fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b64800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa54958c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b64d40 0x55ed92b67130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:40044/0 (socket says 172.21.15.31:40044) 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa543fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b64800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38100/0 (socket says 172.21.15.31:38100) 2024-09-16T00:47:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa54958c640 1 -- 172.21.15.31:0/2753409692 learned_addr learned my addr 172.21.15.31:0/2753409692 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa548d8b640 1 --2- 172.21.15.31:0/2753409692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 0x55ed92b59f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa54958c640 1 -- 172.21.15.31:0/2753409692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 msgr2=0x55ed92b59f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa54958c640 1 --2- 172.21.15.31:0/2753409692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 0x55ed92b59f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa54958c640 1 -- 172.21.15.31:0/2753409692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 msgr2=0x55ed92b64800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa54958c640 1 --2- 172.21.15.31:0/2753409692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b64800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa54958c640 1 -- 172.21.15.31:0/2753409692 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed92a731f0 con 0x55ed92b64d40 2024-09-16T00:47:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa543fff640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa5340a3240 0x7fa5340a5700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa530009fd0 tx=0x7fa530009b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa54958c640 1 --2- 172.21.15.31:0/2753409692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b64d40 0x55ed92b67130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0cd900 tx=0x7fa53c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa592e56e080a613 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa52effd640 1 -- 172.21.15.31:0/2753409692 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c09bd80 con 0x55ed92b64d40 2024-09-16T00:47:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa52effd640 1 -- 172.21.15.31:0/2753409692 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c09cb20 con 0x55ed92b64d40 2024-09-16T00:47:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.501+0000 7fa52effd640 1 -- 172.21.15.31:0/2753409692 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c086ef0 con 0x55ed92b64d40 2024-09-16T00:47:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.502+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2753409692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b64d40 msgr2=0x55ed92b67130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.502+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/2753409692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b64d40 0x55ed92b67130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0cd900 tx=0x7fa53c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.502+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2753409692 shutdown_connections 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.502+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/2753409692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b64d40 0x55ed92b67130 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.502+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/2753409692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b64800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.502+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/2753409692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 0x55ed92b59f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.502+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2753409692 >> 172.21.15.31:0/2753409692 conn(0x55ed92b560f0 msgr2=0x55ed92b564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.502+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2753409692 shutdown_connections 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.502+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2753409692 wait complete. 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54d3c8740 1 -- start start 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54d3c8740 1 Processor -- start 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b59b60 0x55ed92b64800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b61b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 0x55ed92b622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed92b67910 con 0x55ed92b5a480 2024-09-16T00:47:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ed92a79f10 con 0x55ed92b59b60 2024-09-16T00:47:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed92b67a90 con 0x55ed92b64d40 2024-09-16T00:47:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54d3c8740 10 client.0 _ll_register_callbacks cb 0x55ed92a42a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa548d8b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b59b60 0x55ed92b64800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa543fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b61b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa548d8b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b59b60 0x55ed92b64800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:40054/0 (socket says 172.21.15.31:40054) 2024-09-16T00:47:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa548d8b640 1 -- 172.21.15.31:0/411825606 learned_addr learned my addr 172.21.15.31:0/411825606 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.503+0000 7fa54958c640 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 0x55ed92b622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa548d8b640 1 -- 172.21.15.31:0/411825606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 msgr2=0x55ed92b622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa548d8b640 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 0x55ed92b622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa548d8b640 1 -- 172.21.15.31:0/411825606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 msgr2=0x55ed92b61b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa548d8b640 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b61b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa548d8b640 1 -- 172.21.15.31:0/411825606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed928306a0 con 0x55ed92b59b60 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa54958c640 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 0x55ed92b622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa543fff640 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b61b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa548d8b640 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b59b60 0x55ed92b64800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa538000c00 tx=0x7fa538031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97462da0a3cdc06 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa523fff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa523fff640 1 -- 172.21.15.31:0/411825606 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538035040 con 0x55ed92b59b60 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa54d3c8740 1 -- 172.21.15.31:0/411825606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa53c09cb20 con 0x55ed92b59b60 2024-09-16T00:47:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa523fff640 1 -- 172.21.15.31:0/411825606 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa538004240 con 0x55ed92b59b60 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa523fff640 1 -- 172.21.15.31:0/411825606 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa53804f050 con 0x55ed92b59b60 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa54d3c8740 10 client.25781 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.504+0000 7fa54d3c8740 1 -- 172.21.15.31:0/411825606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ed92b73440 con 0x55ed92b59b60 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.505+0000 7fa523fff640 1 -- 172.21.15.31:0/411825606 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c09cb20 con 0x55ed92b59b60 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.505+0000 7fa523fff640 1 -- 172.21.15.31:0/411825606 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fa538046640 con 0x55ed92b59b60 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.505+0000 7fa54d3c8740 10 client.25781 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.505+0000 7fa54d3c8740 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.505+0000 7fa54d3c8740 20 client.25781 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.505+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55ed92b74900 0x55ed92b76d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.505+0000 7fa54d3c8740 4 client.25781 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.505+0000 7fa54d3c8740 1 -- 172.21.15.31:0/411825606 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed92acc7f0 con 0x55ed92b74900 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.505+0000 7fa543fff640 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55ed92b74900 0x55ed92b76d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.506+0000 7fa543fff640 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55ed92b74900 0x55ed92b76d50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd3cc7436da1cfe server_cookie=dcd057a4f11637af in_seq=0 out_seq=0 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.506+0000 7fa523fff640 10 client.25781 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa523fff640 1 -- 172.21.15.31:0/411825606 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed92acc7f0 con 0x55ed92b74900 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa523fff640 10 client.25781 handle_command_reply: tid=0 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 client.25781 shutdown 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 -- 172.21.15.31:0/411825606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b59b60 msgr2=0x55ed92b64800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b59b60 0x55ed92b64800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa538000c00 tx=0x7fa538031850 comp rx=0 tx=0).stop 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 -- 172.21.15.31:0/411825606 shutdown_connections 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55ed92b74900 0x55ed92b76d50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 0x55ed92b622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b5a480 0x55ed92b61b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa523fff640 0 client.25781 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/411825606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b59b60 0x55ed92b64800 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 -- 172.21.15.31:0/411825606 >> 172.21.15.31:0/411825606 conn(0x55ed92b560f0 msgr2=0x55ed92b5be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:07.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 -- 172.21.15.31:0/411825606 shutdown_connections 2024-09-16T00:47:07.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.507+0000 7fa54d3c8740 1 -- 172.21.15.31:0/411825606 wait complete. 2024-09-16T00:47:07.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.508+0000 7fa54d3c8740 20 client.25781 trim_cache size 0 max 16384 2024-09-16T00:47:07.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54d3c8740 1 Processor -- start 2024-09-16T00:47:07.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54d3c8740 1 -- start start 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 0x55ed92b61430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b64d40 0x55ed92b7c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441accd0 con 0x55ed92b64d40 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441af550 con 0x55ed92b5a480 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441ad340 con 0x55ed92b59b60 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54958c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b64d40 0x55ed92b7c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa548d8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 0x55ed92b61430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54958c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b64d40 0x55ed92b7c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38110/0 (socket says 172.21.15.31:38110) 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54958c640 1 -- 172.21.15.31:0/2429491843 learned_addr learned my addr 172.21.15.31:0/2429491843 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa543fff640 1 --2- 172.21.15.31:0/2429491843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa548d8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 0x55ed92b61430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:42432/0 (socket says 172.21.15.31:42432) 2024-09-16T00:47:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54958c640 1 -- 172.21.15.31:0/2429491843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 msgr2=0x55ed92b61430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54958c640 1 --2- 172.21.15.31:0/2429491843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 0x55ed92b61430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54958c640 1 -- 172.21.15.31:0/2429491843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 msgr2=0x55ed92b639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54958c640 1 --2- 172.21.15.31:0/2429491843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa54958c640 1 -- 172.21.15.31:0/2429491843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed92b62650 con 0x55ed92b64d40 2024-09-16T00:47:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.526+0000 7fa543fff640 1 --2- 172.21.15.31:0/2429491843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.527+0000 7fa54958c640 1 --2- 172.21.15.31:0/2429491843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b64d40 0x55ed92b7c620 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0cdc70 tx=0x7fa53c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf266251459844b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.527+0000 7fa52e7fc640 1 -- 172.21.15.31:0/2429491843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0741b0 con 0x55ed92b64d40 2024-09-16T00:47:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.527+0000 7fa52e7fc640 1 -- 172.21.15.31:0/2429491843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c086ef0 con 0x55ed92b64d40 2024-09-16T00:47:07.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.527+0000 7fa52e7fc640 1 -- 172.21.15.31:0/2429491843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0a4e60 con 0x55ed92b64d40 2024-09-16T00:47:07.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2429491843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b64d40 msgr2=0x55ed92b7c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:07.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/2429491843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b64d40 0x55ed92b7c620 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0cdc70 tx=0x7fa53c00cf70 comp rx=0 tx=0).stop 2024-09-16T00:47:07.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2429491843 shutdown_connections 2024-09-16T00:47:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/2429491843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b64d40 0x55ed92b7c620 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/2429491843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/2429491843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b59b60 0x55ed92b61430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2429491843 >> 172.21.15.31:0/2429491843 conn(0x55ed92b560f0 msgr2=0x55ed92b632a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2429491843 shutdown_connections 2024-09-16T00:47:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 -- 172.21.15.31:0/2429491843 wait complete. 2024-09-16T00:47:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 -- start start 2024-09-16T00:47:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.528+0000 7fa54d3c8740 1 Processor -- start 2024-09-16T00:47:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b59b60 0x55ed92b6cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b6d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa54d3c8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 0x55ed92b657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441ada10 con 0x55ed92b59b60 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441aee80 con 0x55ed92b5a480 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa54d3c8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441ae0e0 con 0x55ed92b64d40 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa54d3c8740 10 client.0 _ll_register_callbacks cb 0x7fa538005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa548d8b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b59b60 0x55ed92b6cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa543fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b6d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa543fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b6d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:40076/0 (socket says 172.21.15.31:40076) 2024-09-16T00:47:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa543fff640 1 -- 172.21.15.31:0/4106435065 learned_addr learned my addr 172.21.15.31:0/4106435065 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa54958c640 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 0x55ed92b657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa543fff640 1 -- 172.21.15.31:0/4106435065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 msgr2=0x55ed92b657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa543fff640 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 0x55ed92b657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa543fff640 1 -- 172.21.15.31:0/4106435065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b59b60 msgr2=0x55ed92b6cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa543fff640 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b59b60 0x55ed92b6cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa543fff640 1 -- 172.21.15.31:0/4106435065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed92b65ff0 con 0x55ed92b5a480 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.529+0000 7fa548d8b640 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b59b60 0x55ed92b6cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa543fff640 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b6d4a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa530009900 tx=0x7fa530005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e60c729ade8395 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa523fff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa523fff640 1 -- 172.21.15.31:0/4106435065 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530038060 con 0x55ed92b5a480 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa54d3c8740 1 -- 172.21.15.31:0/4106435065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa53c086ef0 con 0x55ed92b5a480 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa523fff640 1 -- 172.21.15.31:0/4106435065 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa530026040 con 0x55ed92b5a480 2024-09-16T00:47:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa523fff640 1 -- 172.21.15.31:0/4106435065 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa53003f050 con 0x55ed92b5a480 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa54d3c8740 10 client.25787 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa54d3c8740 1 -- 172.21.15.31:0/4106435065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ed92b6c900 con 0x55ed92b5a480 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa523fff640 1 -- 172.21.15.31:0/4106435065 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c086ef0 con 0x55ed92b5a480 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.530+0000 7fa523fff640 1 -- 172.21.15.31:0/4106435065 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fa53002c450 con 0x55ed92b5a480 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.531+0000 7fa54d3c8740 10 client.25787 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.531+0000 7fa54d3c8740 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.531+0000 7fa54d3c8740 20 client.25787 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.531+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55ed92bdf9c0 0x55ed92be1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.531+0000 7fa54d3c8740 4 client.25787 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.531+0000 7fa54d3c8740 1 -- 172.21.15.31:0/4106435065 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ed92ac6d00 con 0x55ed92bdf9c0 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.531+0000 7fa548d8b640 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55ed92bdf9c0 0x55ed92be1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.531+0000 7fa523fff640 10 client.25787 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:07.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.531+0000 7fa548d8b640 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55ed92bdf9c0 0x55ed92be1de0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e0f4664c170cb27 server_cookie=78dfa4e2e20796d9 in_seq=0 out_seq=0 2024-09-16T00:47:07.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa523fff640 1 -- 172.21.15.31:0/4106435065 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ed92ac6d00 con 0x55ed92bdf9c0 2024-09-16T00:47:07.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa523fff640 10 client.25787 handle_command_reply: tid=0 2024-09-16T00:47:07.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 client.25787 shutdown 2024-09-16T00:47:07.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 -- 172.21.15.31:0/4106435065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 msgr2=0x55ed92b6d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:07.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b6d4a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa530009900 tx=0x7fa530005f90 comp rx=0 tx=0).stop 2024-09-16T00:47:07.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 -- 172.21.15.31:0/4106435065 shutdown_connections 2024-09-16T00:47:07.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55ed92bdf9c0 0x55ed92be1de0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed92b64d40 0x55ed92b657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed92b5a480 0x55ed92b6d4a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 --2- 172.21.15.31:0/4106435065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed92b59b60 0x55ed92b6cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 -- 172.21.15.31:0/4106435065 >> 172.21.15.31:0/4106435065 conn(0x55ed92b560f0 msgr2=0x55ed92b5bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:07.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa523fff640 0 client.25787 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:07.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 -- 172.21.15.31:0/4106435065 shutdown_connections 2024-09-16T00:47:07.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.537+0000 7fa54d3c8740 1 -- 172.21.15.31:0/4106435065 wait complete. 2024-09-16T00:47:07.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54d3c8740 20 client.25787 trim_cache size 0 max 16384 2024-09-16T00:47:07.541 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:47:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:47:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "a407dc65-2131-4524-990e-434dacc82221", 2024-09-16T00:47:07.541 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:47:07.541 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 -- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa5340a3240 msgr2=0x7fa5340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa5340a3240 0x7fa5340a5700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa530009fd0 tx=0x7fa530009b80 comp rx=0 tx=0).stop 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 -- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5441082a0 msgr2=0x7fa54419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5441082a0 0x7fa54419ce90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00eb10 tx=0x7fa53c00c2d0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 -- 172.21.15.31:0/246397701 shutdown_connections 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5441082a0 0x7fa54419ce90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5441036e0 0x7fa544198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa5340a3240 0x7fa5340a5700 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 --2- 172.21.15.31:0/246397701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa544102d30 0x7fa5441985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:07.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 -- 172.21.15.31:0/246397701 >> 172.21.15.31:0/246397701 conn(0x7fa5440fc7d0 msgr2=0x7fa544105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:07.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 -- 172.21.15.31:0/246397701 shutdown_connections 2024-09-16T00:47:07.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:07.538+0000 7fa54afef640 1 -- 172.21.15.31:0/246397701 wait complete. 2024-09-16T00:47:08.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:47:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f960ffff640 1 -- 172.21.15.31:0/2570103861 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a083f33f40 con 0x55a0840d6840 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f960ffff640 10 client.25727 handle_command_reply: tid=0 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 client.25727 shutdown 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 -- 172.21.15.31:0/2570103861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 msgr2=0x55a08405f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08405f220 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f962401d5b0 tx=0x7f9624004580 comp rx=0 tx=0).stop 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 -- 172.21.15.31:0/2570103861 shutdown_connections 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a08405ae40 0x55a0840d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55a0840d6840 0x55a0840d8c60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a08404c050 0x55a08405f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 --2- 172.21.15.31:0/2570103861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a08404b730 0x55a08405f220 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f960ffff640 0 client.25727 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 -- 172.21.15.31:0/2570103861 >> 172.21.15.31:0/2570103861 conn(0x55a084047de0 msgr2=0x55a084046ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 -- 172.21.15.31:0/2570103861 shutdown_connections 2024-09-16T00:47:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 1 -- 172.21.15.31:0/2570103861 wait complete. 2024-09-16T00:47:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.894+0000 7f9639078740 20 client.25727 trim_cache size 0 max 16384 2024-09-16T00:47:08.897 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:47:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:47:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:47:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 22, 2024-09-16T00:47:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:47:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "9da9a417": { 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 22, 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 145.3, 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "QUIESCED", 2024-09-16T00:47:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:47:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:08.899 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:47:08.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 -- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96200a3240 msgr2=0x7f96200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:08.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96200a3240 0x7f96200a5700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9624009fc0 tx=0x7f9624005f90 comp rx=0 tx=0).stop 2024-09-16T00:47:08.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 -- 172.21.15.31:0/1923688176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9630102570 msgr2=0x7f9630198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:08.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9630102570 0x7f9630198a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f961c02a9a0 tx=0x7f961c003b20 comp rx=0 tx=0).stop 2024-09-16T00:47:08.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 -- 172.21.15.31:0/1923688176 shutdown_connections 2024-09-16T00:47:08.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9630102ef0 0x7f963019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:08.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9630102570 0x7f9630198a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:08.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f96200a3240 0x7f96200a5700 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 --2- 172.21.15.31:0/1923688176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9630100e20 0x7f9630198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 -- 172.21.15.31:0/1923688176 >> 172.21.15.31:0/1923688176 conn(0x7f96300fc7d0 msgr2=0x7f963006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 -- 172.21.15.31:0/1923688176 shutdown_connections 2024-09-16T00:47:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:08.895+0000 7f9636c9f640 1 -- 172.21.15.31:0/1923688176 wait complete. 2024-09-16T00:47:09.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:08 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:47:09.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:08 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:47:09.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:08 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:47:09.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:08 smithi031 ceph-mon[30956]: pgmap v536: 129 pgs: 129 active+clean; 617 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 793 KiB/s wr, 329 op/s 2024-09-16T00:47:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:08 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:47:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:08 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:47:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:08 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:47:09.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:08 smithi081 ceph-mon[35539]: pgmap v536: 129 pgs: 129 active+clean; 617 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 793 KiB/s wr, 329 op/s 2024-09-16T00:47:09.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:08 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:47:09.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:08 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:47:09.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:08 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:47:09.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:08 smithi102 ceph-mon[35364]: pgmap v536: 129 pgs: 129 active+clean; 617 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 793 KiB/s wr, 329 op/s 2024-09-16T00:47:09.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 -- 172.21.15.31:0/3726387874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60a0108290 msgr2=0x7f60a010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:09.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/3726387874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60a0108290 0x7f60a010ec30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6098009cd0 tx=0x7f609802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 -- 172.21.15.31:0/3726387874 shutdown_connections 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/3726387874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60a0108290 0x7f60a010ec30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/3726387874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a0103740 0x7f60a0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/3726387874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60a0102d90 0x7f60a0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 -- 172.21.15.31:0/3726387874 >> 172.21.15.31:0/3726387874 conn(0x7f60a00fc7d0 msgr2=0x7f60a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 -- 172.21.15.31:0/3726387874 shutdown_connections 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 -- 172.21.15.31:0/3726387874 wait complete. 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.367+0000 7f60a6a0b640 1 Processor -- start 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f60a6a0b640 1 -- start start 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f60a6a0b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a0102d90 0x7f60a0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f60a6a0b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60a0103740 0x7f60a0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f60a6a0b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60a0108290 0x7f60a019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f60a6a0b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0112090 con 0x7f60a0103740 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f60a6a0b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0111f10 con 0x7f60a0108290 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f60a6a0b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0112210 con 0x7f60a0102d90 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f609ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a0102d90 0x7f60a0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f609f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60a0103740 0x7f60a0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f609ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a0102d90 0x7f60a0198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:42456/0 (socket says 172.21.15.31:42456) 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f609ffff640 1 -- 172.21.15.31:0/289939487 learned_addr learned my addr 172.21.15.31:0/289939487 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f60a4fa8640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60a0108290 0x7f60a019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f609ffff640 1 -- 172.21.15.31:0/289939487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60a0108290 msgr2=0x7f60a019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f609ffff640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60a0108290 0x7f60a019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f609ffff640 1 -- 172.21.15.31:0/289939487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60a0103740 msgr2=0x7f60a0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.368+0000 7f609ffff640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60a0103740 0x7f60a0198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.369+0000 7f609ffff640 1 -- 172.21.15.31:0/289939487 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a019d500 con 0x7f60a0102d90 2024-09-16T00:47:09.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.369+0000 7f609ffff640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a0102d90 0x7f60a0198550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f609000c600 tx=0x7f609000cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:09.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.369+0000 7f609d7fa640 1 -- 172.21.15.31:0/289939487 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090023070 con 0x7f60a0102d90 2024-09-16T00:47:09.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.369+0000 7f60a6a0b640 1 -- 172.21.15.31:0/289939487 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a019d7f0 con 0x7f60a0102d90 2024-09-16T00:47:09.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.369+0000 7f609d7fa640 1 -- 172.21.15.31:0/289939487 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609001e910 con 0x7f60a0102d90 2024-09-16T00:47:09.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.369+0000 7f609d7fa640 1 -- 172.21.15.31:0/289939487 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609000bae0 con 0x7f60a0102d90 2024-09-16T00:47:09.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.369+0000 7f60a6a0b640 1 -- 172.21.15.31:0/289939487 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a0100920 con 0x7f60a0102d90 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.372+0000 7f60a8de4740 1 Processor -- start 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.372+0000 7f609d7fa640 1 -- 172.21.15.31:0/289939487 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6090026070 con 0x7f60a0102d90 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f60a8de4740 1 -- start start 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 0x564921575880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x564921580140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921582a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f60a8de4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5649214db2e0 con 0x5649215754a0 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f60a8de4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564921482ce0 con 0x564921575dc0 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f60a8de4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5649214d72a0 con 0x564921580680 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f609f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921582a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f609ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x564921580140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f60a4fa8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 0x564921575880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f609f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921582a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:42464/0 (socket says 172.21.15.31:42464) 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f609f7fe640 1 -- 172.21.15.31:0/3249492306 learned_addr learned my addr 172.21.15.31:0/3249492306 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f609d7fa640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60880a32c0 0x7f60880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.373+0000 7f60a4fa8640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60880a32c0 0x7f60880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f609f7fe640 1 -- 172.21.15.31:0/3249492306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 msgr2=0x564921580140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f609f7fe640 1 --2- 172.21.15.31:0/3249492306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x564921580140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f609d7fa640 1 -- 172.21.15.31:0/289939487 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f60900d7e80 con 0x7f60a0102d90 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f609f7fe640 1 -- 172.21.15.31:0/3249492306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 msgr2=0x564921575880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f609f7fe640 1 --2- 172.21.15.31:0/3249492306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 0x564921575880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f609f7fe640 1 -- 172.21.15.31:0/3249492306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564921467440 con 0x564921580680 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f609f7fe640 1 --2- 172.21.15.31:0/3249492306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921582a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6094010a90 tx=0x7f6094005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcfb57b73eb65a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:09.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f60867fc640 1 -- 172.21.15.31:0/3249492306 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094004050 con 0x564921580680 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f60a4fa8640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60880a32c0 0x7f60880a5780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6098005fe0 tx=0x7f6098005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f60867fc640 1 -- 172.21.15.31:0/3249492306 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6094005500 con 0x564921580680 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.374+0000 7f60867fc640 1 -- 172.21.15.31:0/3249492306 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094004c80 con 0x564921580680 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.375+0000 7f60a8de4740 1 -- 172.21.15.31:0/3249492306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 msgr2=0x564921582a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.375+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3249492306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921582a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6094010a90 tx=0x7f6094005f90 comp rx=0 tx=0).stop 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.375+0000 7f60a8de4740 1 -- 172.21.15.31:0/3249492306 shutdown_connections 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.375+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3249492306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921582a70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.375+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3249492306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x564921580140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.375+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3249492306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 0x564921575880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.375+0000 7f60a8de4740 1 -- 172.21.15.31:0/3249492306 >> 172.21.15.31:0/3249492306 conn(0x564921571a30 msgr2=0x564921571e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.375+0000 7f60a8de4740 1 -- 172.21.15.31:0/3249492306 shutdown_connections 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.375+0000 7f60a8de4740 1 -- 172.21.15.31:0/3249492306 wait complete. 2024-09-16T00:47:09.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.376+0000 7f60a8de4740 1 -- start start 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.376+0000 7f60a8de4740 1 Processor -- start 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.376+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5649215754a0 0x56492157ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.376+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x56492157d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.376+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921580680 0x56492157d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.376+0000 7f60a8de4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564921583250 con 0x564921580680 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.376+0000 7f60a8de4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564921496050 con 0x564921575dc0 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.376+0000 7f60a8de4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5649215833d0 con 0x5649215754a0 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.376+0000 7f60a8de4740 10 client.0 _ll_register_callbacks cb 0x5649214d6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5649215754a0 0x56492157ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x56492157d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5649215754a0 0x56492157ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:42478/0 (socket says 172.21.15.31:42478) 2024-09-16T00:47:09.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609ffff640 1 -- 172.21.15.31:0/573242841 learned_addr learned my addr 172.21.15.31:0/573242841 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f60a4fa8640 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921580680 0x56492157d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609ffff640 1 -- 172.21.15.31:0/573242841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 msgr2=0x56492157d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609ffff640 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x56492157d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609ffff640 1 -- 172.21.15.31:0/573242841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921580680 msgr2=0x56492157d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609ffff640 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921580680 0x56492157d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609ffff640 1 -- 172.21.15.31:0/573242841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56492124c3c0 con 0x5649215754a0 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f609ffff640 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5649215754a0 0x56492157ff80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f60900a05a0 tx=0x7f60900a3d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e6203663034472 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f607f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.377+0000 7f607f7fe640 1 -- 172.21.15.31:0/573242841 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900acd00 con 0x5649215754a0 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f607f7fe640 1 -- 172.21.15.31:0/573242841 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f60900a80d0 con 0x5649215754a0 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f60a8de4740 1 -- 172.21.15.31:0/573242841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6094005500 con 0x5649215754a0 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f607f7fe640 1 -- 172.21.15.31:0/573242841 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f60900a07b0 con 0x5649215754a0 2024-09-16T00:47:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f60a8de4740 10 client.25726 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f60a8de4740 1 -- 172.21.15.31:0/573242841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56492158ebc0 con 0x5649215754a0 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f607f7fe640 1 -- 172.21.15.31:0/573242841 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6094005500 con 0x5649215754a0 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f607f7fe640 1 -- 172.21.15.31:0/573242841 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f60900a36b0 con 0x5649215754a0 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f60a8de4740 10 client.25726 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f60a8de4740 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f60a8de4740 20 client.25726 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f60a8de4740 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56492158ffd0 0x5649215923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f60a8de4740 4 client.25726 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f60a8de4740 1 -- 172.21.15.31:0/573242841 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649214e8850 con 0x56492158ffd0 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.378+0000 7f609f7fe640 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56492158ffd0 0x5649215923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.379+0000 7f607f7fe640 10 client.25726 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:09.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.379+0000 7f609f7fe640 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56492158ffd0 0x5649215923e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f37001afaf411712 server_cookie=bb9d4517b5d61bd in_seq=0 out_seq=0 2024-09-16T00:47:09.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f607f7fe640 1 -- 172.21.15.31:0/573242841 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649214e8850 con 0x56492158ffd0 2024-09-16T00:47:09.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f607f7fe640 10 client.25726 handle_command_reply: tid=0 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f60a8de4740 1 client.25726 shutdown 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f60a8de4740 1 -- 172.21.15.31:0/573242841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5649215754a0 msgr2=0x56492157ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f60a8de4740 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5649215754a0 0x56492157ff80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f60900a05a0 tx=0x7f60900a3d00 comp rx=0 tx=0).stop 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f60a8de4740 1 -- 172.21.15.31:0/573242841 shutdown_connections 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f60a8de4740 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921580680 0x56492157d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f60a8de4740 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x56492157d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f60a8de4740 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56492158ffd0 0x5649215923e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.380+0000 7f60a8de4740 1 --2- 172.21.15.31:0/573242841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5649215754a0 0x56492157ff80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.381+0000 7f607f7fe640 0 client.25726 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.381+0000 7f60a8de4740 1 -- 172.21.15.31:0/573242841 >> 172.21.15.31:0/573242841 conn(0x564921571a30 msgr2=0x564921577760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.381+0000 7f60a8de4740 1 -- 172.21.15.31:0/573242841 shutdown_connections 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.381+0000 7f60a8de4740 1 -- 172.21.15.31:0/573242841 wait complete. 2024-09-16T00:47:09.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.381+0000 7f60a8de4740 20 client.25726 trim_cache size 0 max 16384 2024-09-16T00:47:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f60a8de4740 1 Processor -- start 2024-09-16T00:47:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f60a8de4740 1 -- start start 2024-09-16T00:47:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649215754a0 0x56492158c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921575dc0 0x564921597c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921598190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f60a8de4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01aaac0 con 0x564921575dc0 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f60a8de4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01ad340 con 0x5649215754a0 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f60a8de4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01ab130 con 0x564921580680 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f609f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921575dc0 0x564921597c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f609f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921575dc0 0x564921597c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38172/0 (socket says 172.21.15.31:38172) 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f609f7fe640 1 -- 172.21.15.31:0/2653691451 learned_addr learned my addr 172.21.15.31:0/2653691451 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f609ffff640 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649215754a0 0x56492158c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f60a4fa8640 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921598190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f609ffff640 1 -- 172.21.15.31:0/2653691451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 msgr2=0x564921598190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f609ffff640 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921598190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f609ffff640 1 -- 172.21.15.31:0/2653691451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921575dc0 msgr2=0x564921597c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:09.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.400+0000 7f609ffff640 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921575dc0 0x564921597c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.401+0000 7f609ffff640 1 -- 172.21.15.31:0/2653691451 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564921575220 con 0x5649215754a0 2024-09-16T00:47:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.401+0000 7f609f7fe640 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921575dc0 0x564921597c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.401+0000 7f609ffff640 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649215754a0 0x56492158c4b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f60900c4010 tx=0x7f60900a2650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb748b2d32ec05a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.401+0000 7f6085ffb640 1 -- 172.21.15.31:0/2653691451 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900a80d0 con 0x5649215754a0 2024-09-16T00:47:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.401+0000 7f6085ffb640 1 -- 172.21.15.31:0/2653691451 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609000bc80 con 0x5649215754a0 2024-09-16T00:47:09.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.401+0000 7f6085ffb640 1 -- 172.21.15.31:0/2653691451 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900a37f0 con 0x5649215754a0 2024-09-16T00:47:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 -- 172.21.15.31:0/2653691451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649215754a0 msgr2=0x56492158c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649215754a0 0x56492158c4b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f60900c4010 tx=0x7f60900a2650 comp rx=0 tx=0).stop 2024-09-16T00:47:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 -- 172.21.15.31:0/2653691451 shutdown_connections 2024-09-16T00:47:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921598190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564921575dc0 0x564921597c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 --2- 172.21.15.31:0/2653691451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649215754a0 0x56492158c4b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 -- 172.21.15.31:0/2653691451 >> 172.21.15.31:0/2653691451 conn(0x564921571a30 msgr2=0x564921578850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 -- 172.21.15.31:0/2653691451 shutdown_connections 2024-09-16T00:47:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 -- 172.21.15.31:0/2653691451 wait complete. 2024-09-16T00:47:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 -- start start 2024-09-16T00:47:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.402+0000 7f60a8de4740 1 Processor -- start 2024-09-16T00:47:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 0x564921584510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x564921584a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f60a8de4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921584f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f60a8de4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01ab800 con 0x5649215754a0 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f60a8de4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01acc70 con 0x564921575dc0 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f60a8de4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01abed0 con 0x564921580680 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f60a8de4740 10 client.0 _ll_register_callbacks cb 0x7f609000c550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f609ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 0x564921584510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f609ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 0x564921584510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38180/0 (socket says 172.21.15.31:38180) 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f609f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x564921584a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f609ffff640 1 -- 172.21.15.31:0/3756754368 learned_addr learned my addr 172.21.15.31:0/3756754368 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f60a4fa8640 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921584f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f609f7fe640 1 -- 172.21.15.31:0/3756754368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 msgr2=0x564921584f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f609f7fe640 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921584f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f609f7fe640 1 -- 172.21.15.31:0/3756754368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 msgr2=0x564921584510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f609f7fe640 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 0x564921584510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.403+0000 7f609f7fe640 1 -- 172.21.15.31:0/3756754368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649215f9d80 con 0x564921575dc0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f609f7fe640 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x564921584a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f60940058c0 tx=0x7f60940048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=736e5592d3b0b424 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f607f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f607f7fe640 1 -- 172.21.15.31:0/3756754368 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094009dd0 con 0x564921575dc0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f607f7fe640 1 -- 172.21.15.31:0/3756754368 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f6094004050 con 0x564921575dc0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f60a8de4740 1 -- 172.21.15.31:0/3756754368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f609000bc80 con 0x564921575dc0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f607f7fe640 1 -- 172.21.15.31:0/3756754368 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f609403e050 con 0x564921575dc0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f60a8de4740 10 client.25805 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f60a8de4740 1 -- 172.21.15.31:0/3756754368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5649215fb050 con 0x564921575dc0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f607f7fe640 1 -- 172.21.15.31:0/3756754368 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609000bc80 con 0x564921575dc0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f607f7fe640 1 -- 172.21.15.31:0/3756754368 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f6094018440 con 0x564921575dc0 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f60a8de4740 10 client.25805 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.404+0000 7f60a8de4740 10 client.25805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.405+0000 7f60a8de4740 20 client.25805 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.405+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5649215fb4f0 0x5649215fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.405+0000 7f60a8de4740 4 client.25805 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:47:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.405+0000 7f60a8de4740 1 -- 172.21.15.31:0/3756754368 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56492157db00 con 0x5649215fb4f0 2024-09-16T00:47:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.405+0000 7f609ffff640 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5649215fb4f0 0x5649215fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.405+0000 7f609ffff640 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5649215fb4f0 0x5649215fd8e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118b010433b3aacd server_cookie=1aa64441c8fdde11 in_seq=0 out_seq=0 2024-09-16T00:47:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.405+0000 7f607f7fe640 10 client.25805 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f607f7fe640 1 -- 172.21.15.31:0/3756754368 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x56492157db00 con 0x5649215fb4f0 2024-09-16T00:47:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f607f7fe640 10 client.25805 handle_command_reply: tid=0 2024-09-16T00:47:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 client.25805 shutdown 2024-09-16T00:47:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 -- 172.21.15.31:0/3756754368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 msgr2=0x564921584a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x564921584a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f60940058c0 tx=0x7f60940048e0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 -- 172.21.15.31:0/3756754368 shutdown_connections 2024-09-16T00:47:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5649215fb4f0 0x5649215fd8e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564921580680 0x564921584f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564921575dc0 0x564921584a50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 --2- 172.21.15.31:0/3756754368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649215754a0 0x564921584510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 -- 172.21.15.31:0/3756754368 >> 172.21.15.31:0/3756754368 conn(0x564921571a30 msgr2=0x564921576ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f607f7fe640 0 client.25805 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 -- 172.21.15.31:0/3756754368 shutdown_connections 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 1 -- 172.21.15.31:0/3756754368 wait complete. 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.407+0000 7f60a8de4740 20 client.25805 trim_cache size 0 max 16384 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:47:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (19508 inodes in the stack)", 2024-09-16T00:47:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:47:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "a407dc65-2131-4524-990e-434dacc82221": { 2024-09-16T00:47:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:47:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "a407dc65-2131-4524-990e-434dacc82221", 2024-09-16T00:47:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:47:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:09.413 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:47:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.408+0000 7f60a6a0b640 1 -- 172.21.15.31:0/289939487 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60880a32c0 msgr2=0x7f60880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.408+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60880a32c0 0x7f60880a5780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6098005fe0 tx=0x7f6098005f50 comp rx=0 tx=0).stop 2024-09-16T00:47:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.408+0000 7f60a6a0b640 1 -- 172.21.15.31:0/289939487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a0102d90 msgr2=0x7f60a0198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.408+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a0102d90 0x7f60a0198550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f609000c600 tx=0x7f609000cac0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.409+0000 7f60a6a0b640 1 -- 172.21.15.31:0/289939487 shutdown_connections 2024-09-16T00:47:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.409+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60a0108290 0x7f60a019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.409+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60a0103740 0x7f60a0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.409+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f60880a32c0 0x7f60880a5780 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.409+0000 7f60a6a0b640 1 --2- 172.21.15.31:0/289939487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a0102d90 0x7f60a0198550 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.409+0000 7f60a6a0b640 1 -- 172.21.15.31:0/289939487 >> 172.21.15.31:0/289939487 conn(0x7f60a00fc7d0 msgr2=0x7f60a0106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.409+0000 7f60a6a0b640 1 -- 172.21.15.31:0/289939487 shutdown_connections 2024-09-16T00:47:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:09.409+0000 7f60a6a0b640 1 -- 172.21.15.31:0/289939487 wait complete. 2024-09-16T00:47:09.859 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9da9a417', quiesce took 15.2 seconds. Will release after: 10.150000000000002 2024-09-16T00:47:10.680 INFO:tasks.cephfs.filesystem:scrub status for tag:a407dc65-2131-4524-990e-434dacc82221 - {'path': '/', 'tag': 'a407dc65-2131-4524-990e-434dacc82221', 'options': 'recursive,force'} 2024-09-16T00:47:11.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:10 smithi081 ceph-mon[35539]: pgmap v537: 129 pgs: 129 active+clean; 618 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 913 KiB/s wr, 332 op/s 2024-09-16T00:47:11.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:10 smithi102 ceph-mon[35364]: pgmap v537: 129 pgs: 129 active+clean; 618 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 913 KiB/s wr, 332 op/s 2024-09-16T00:47:11.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:10 smithi031 ceph-mon[30956]: pgmap v537: 129 pgs: 129 active+clean; 618 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 913 KiB/s wr, 332 op/s 2024-09-16T00:47:13.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:12 smithi081 ceph-mon[35539]: pgmap v538: 129 pgs: 129 active+clean; 621 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 965 KiB/s wr, 393 op/s 2024-09-16T00:47:13.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:12 smithi102 ceph-mon[35364]: pgmap v538: 129 pgs: 129 active+clean; 621 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 965 KiB/s wr, 393 op/s 2024-09-16T00:47:13.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:12 smithi031 ceph-mon[30956]: pgmap v538: 129 pgs: 129 active+clean; 621 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 965 KiB/s wr, 393 op/s 2024-09-16T00:47:15.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:14 smithi081 ceph-mon[35539]: pgmap v539: 129 pgs: 129 active+clean; 624 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 613 op/s 2024-09-16T00:47:15.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:14 smithi102 ceph-mon[35364]: pgmap v539: 129 pgs: 129 active+clean; 624 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 613 op/s 2024-09-16T00:47:15.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:14 smithi031 ceph-mon[30956]: pgmap v539: 129 pgs: 129 active+clean; 624 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 613 op/s 2024-09-16T00:47:16.190 INFO:tasks.workunit.client.0.smithi031.stdout: 117 6246 0 4088 0 3516 0 2024-09-16T00:47:17.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:16 smithi081 ceph-mon[35539]: pgmap v540: 129 pgs: 129 active+clean; 624 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.1 MiB/s wr, 672 op/s 2024-09-16T00:47:17.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:16 smithi102 ceph-mon[35364]: pgmap v540: 129 pgs: 129 active+clean; 624 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.1 MiB/s wr, 672 op/s 2024-09-16T00:47:17.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:16 smithi031 ceph-mon[30956]: pgmap v540: 129 pgs: 129 active+clean; 624 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.1 MiB/s wr, 672 op/s 2024-09-16T00:47:19.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:18 smithi081 ceph-mon[35539]: pgmap v541: 129 pgs: 129 active+clean; 608 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 902 KiB/s wr, 829 op/s 2024-09-16T00:47:19.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:18 smithi102 ceph-mon[35364]: pgmap v541: 129 pgs: 129 active+clean; 608 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 902 KiB/s wr, 829 op/s 2024-09-16T00:47:19.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:18 smithi031 ceph-mon[30956]: pgmap v541: 129 pgs: 129 active+clean; 608 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 902 KiB/s wr, 829 op/s 2024-09-16T00:47:20.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:47:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:21.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:20 smithi031 ceph-mon[30956]: pgmap v542: 129 pgs: 129 active+clean; 597 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 784 KiB/s wr, 989 op/s 2024-09-16T00:47:21.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:20 smithi081 ceph-mon[35539]: pgmap v542: 129 pgs: 129 active+clean; 597 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 784 KiB/s wr, 989 op/s 2024-09-16T00:47:21.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:20 smithi102 ceph-mon[35364]: pgmap v542: 129 pgs: 129 active+clean; 597 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 784 KiB/s wr, 989 op/s 2024-09-16T00:47:21.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.331+0000 7f9b0ce23640 1 -- 172.21.15.31:0/322788448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b08102de0 msgr2=0x7f9b081031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:21.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.331+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/322788448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b08102de0 0x7f9b081031c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009900 tx=0x7f9af802a480 comp rx=0 tx=0).stop 2024-09-16T00:47:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.331+0000 7f9b0ce23640 1 -- 172.21.15.31:0/322788448 shutdown_connections 2024-09-16T00:47:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.331+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/322788448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b0810c180 0x7f9b0810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.331+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/322788448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b08103700 0x7f9b0810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.331+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/322788448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b08102de0 0x7f9b081031c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.331+0000 7f9b0ce23640 1 -- 172.21.15.31:0/322788448 >> 172.21.15.31:0/322788448 conn(0x7f9b080fc780 msgr2=0x7f9b080feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:21.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.331+0000 7f9b0ce23640 1 -- 172.21.15.31:0/322788448 shutdown_connections 2024-09-16T00:47:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.331+0000 7f9b0ce23640 1 -- 172.21.15.31:0/322788448 wait complete. 2024-09-16T00:47:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b0ce23640 1 Processor -- start 2024-09-16T00:47:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b0ce23640 1 -- start start 2024-09-16T00:47:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b0ce23640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b08102de0 0x7f9b08198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b0ce23640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b08103700 0x7f9b08198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b0ce23640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b0810c180 0x7f9b0819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b0ce23640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08111fc0 con 0x7f9b0810c180 2024-09-16T00:47:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b0ce23640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08111e40 con 0x7f9b08102de0 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b0ce23640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08112140 con 0x7f9b08103700 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b0810c180 0x7f9b0819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b0659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b08102de0 0x7f9b08198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b0810c180 0x7f9b0819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54540/0 (socket says 172.21.15.31:54540) 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b06d9d640 1 -- 172.21.15.31:0/921282395 learned_addr learned my addr 172.21.15.31:0/921282395 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b05d9b640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b08103700 0x7f9b08198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b06d9d640 1 -- 172.21.15.31:0/921282395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b08103700 msgr2=0x7f9b08198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b06d9d640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b08103700 0x7f9b08198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.332+0000 7f9b06d9d640 1 -- 172.21.15.31:0/921282395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b08102de0 msgr2=0x7f9b08198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.333+0000 7f9b06d9d640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b08102de0 0x7f9b08198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.333+0000 7f9b06d9d640 1 -- 172.21.15.31:0/921282395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0819d510 con 0x7f9b0810c180 2024-09-16T00:47:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.333+0000 7f9b06d9d640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b0810c180 0x7f9b0819ce30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000eb10 tx=0x7f9b0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.333+0000 7f9aff7fe640 1 -- 172.21.15.31:0/921282395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00004b20 con 0x7f9b0810c180 2024-09-16T00:47:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.333+0000 7f9aff7fe640 1 -- 172.21.15.31:0/921282395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b00004cc0 con 0x7f9b0810c180 2024-09-16T00:47:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.333+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0819d800 con 0x7f9b0810c180 2024-09-16T00:47:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.333+0000 7f9aff7fe640 1 -- 172.21.15.31:0/921282395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00010760 con 0x7f9b0810c180 2024-09-16T00:47:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.333+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b081a50e0 con 0x7f9b0810c180 2024-09-16T00:47:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.335+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b081031c0 con 0x7f9b0810c180 2024-09-16T00:47:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.336+0000 7f9aff7fe640 1 -- 172.21.15.31:0/921282395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9b00005020 con 0x7f9b0810c180 2024-09-16T00:47:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.336+0000 7f9aff7fe640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9af40a32c0 0x7f9af40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.337+0000 7f9b0659c640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9af40a32c0 0x7f9af40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.337+0000 7f9aff7fe640 1 -- 172.21.15.31:0/921282395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f9b000d0150 con 0x7f9b0810c180 2024-09-16T00:47:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.337+0000 7f9b0659c640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9af40a32c0 0x7f9af40a5780 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9af8000c00 tx=0x7f9af8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.341+0000 7f9aff7fe640 1 -- 172.21.15.31:0/921282395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b0009cac0 con 0x7f9b0810c180 2024-09-16T00:47:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.583+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b08107260 con 0x7f9b0810c180 2024-09-16T00:47:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.584+0000 7f9aff7fe640 1 -- 172.21.15.31:0/921282395 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f9b0009c3c0 con 0x7f9b0810c180 2024-09-16T00:47:21.584 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:47:21.584 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:47:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9af40a32c0 msgr2=0x7f9af40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9af40a32c0 0x7f9af40a5780 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9af8000c00 tx=0x7f9af8005b50 comp rx=0 tx=0).stop 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b0810c180 msgr2=0x7f9b0819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b0810c180 0x7f9b0819ce30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000eb10 tx=0x7f9b0000c2d0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 shutdown_connections 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b0810c180 0x7f9b0819ce30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9af40a32c0 0x7f9af40a5780 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b08103700 0x7f9b08198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 --2- 172.21.15.31:0/921282395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b08102de0 0x7f9b08198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 >> 172.21.15.31:0/921282395 conn(0x7f9b080fc780 msgr2=0x7f9b080fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 shutdown_connections 2024-09-16T00:47:21.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:21.586+0000 7f9b0ce23640 1 -- 172.21.15.31:0/921282395 wait complete. 2024-09-16T00:47:22.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:21 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/921282395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:47:22.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:21 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/921282395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:47:22.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:21 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/921282395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:47:22.542 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 9da9a417 --release --await' 2024-09-16T00:47:22.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 9da9a417 --release --await 2024-09-16T00:47:22.828 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:23.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:22 smithi081 ceph-mon[35539]: pgmap v543: 129 pgs: 129 active+clean; 601 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 973 KiB/s wr, 1.10k op/s 2024-09-16T00:47:23.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:22 smithi031 ceph-mon[30956]: pgmap v543: 129 pgs: 129 active+clean; 601 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 973 KiB/s wr, 1.10k op/s 2024-09-16T00:47:23.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:22 smithi102 ceph-mon[35364]: pgmap v543: 129 pgs: 129 active+clean; 601 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 973 KiB/s wr, 1.10k op/s 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.374+0000 7f31c5c0c640 1 -- 172.21.15.31:0/2190098788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0106f10 msgr2=0x7f31c010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.374+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/2190098788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0106f10 0x7f31c010ea90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009ca0 tx=0x7f31b802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.374+0000 7f31c5c0c640 1 -- 172.21.15.31:0/2190098788 shutdown_connections 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.374+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/2190098788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0106f10 0x7f31c010ea90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.374+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/2190098788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31c0102790 0x7f31c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.374+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/2190098788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31c0101de0 0x7f31c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.374+0000 7f31c5c0c640 1 -- 172.21.15.31:0/2190098788 >> 172.21.15.31:0/2190098788 conn(0x7f31c00fd8c0 msgr2=0x7f31c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.374+0000 7f31c5c0c640 1 -- 172.21.15.31:0/2190098788 shutdown_connections 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.374+0000 7f31c5c0c640 1 -- 172.21.15.31:0/2190098788 wait complete. 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.375+0000 7f31c5c0c640 1 Processor -- start 2024-09-16T00:47:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.375+0000 7f31c5c0c640 1 -- start start 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.375+0000 7f31c5c0c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0101de0 0x7f31c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.375+0000 7f31c5c0c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31c0102790 0x7f31c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.375+0000 7f31c5c0c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31c0106f10 0x7f31c0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.375+0000 7f31c5c0c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f31c0111f70 con 0x7f31c0101de0 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.375+0000 7f31c5c0c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f31c0111df0 con 0x7f31c0102790 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.375+0000 7f31c5c0c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31c01120f0 con 0x7f31c0106f10 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0101de0 0x7f31c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0101de0 0x7f31c00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54548/0 (socket says 172.21.15.31:54548) 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bf7fe640 1 -- 172.21.15.31:0/73085807 learned_addr learned my addr 172.21.15.31:0/73085807 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31beffd640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31c0102790 0x7f31c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bffff640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31c0106f10 0x7f31c0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bf7fe640 1 -- 172.21.15.31:0/73085807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31c0106f10 msgr2=0x7f31c0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31c0106f10 0x7f31c0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bf7fe640 1 -- 172.21.15.31:0/73085807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31c0102790 msgr2=0x7f31c0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31c0102790 0x7f31c0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bf7fe640 1 -- 172.21.15.31:0/73085807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c01a6870 con 0x7f31c0101de0 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0101de0 0x7f31c00744f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f31b000e9f0 tx=0x7f31b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bcff9640 1 -- 172.21.15.31:0/73085807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b000cd80 con 0x7f31c0101de0 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bcff9640 1 -- 172.21.15.31:0/73085807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b0010910 con 0x7f31c0101de0 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31c5c0c640 1 -- 172.21.15.31:0/73085807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c01a6b60 con 0x7f31c0101de0 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31bcff9640 1 -- 172.21.15.31:0/73085807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b0018990 con 0x7f31c0101de0 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.376+0000 7f31c5c0c640 1 -- 172.21.15.31:0/73085807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c01a70f0 con 0x7f31c0101de0 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.379+0000 7f31bcff9640 1 -- 172.21.15.31:0/73085807 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f31b0010ac0 con 0x7f31c0101de0 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.379+0000 7f31c7fe5740 1 Processor -- start 2024-09-16T00:47:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.379+0000 7f31c7fe5740 1 -- start start 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.379+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3615440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.379+0000 7f31bcff9640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f31a80a32c0 0x7f31a80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3620240 0x563da3622630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563da357a620 con 0x563da3615980 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563da3521e50 con 0x563da3615060 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563da35765e0 con 0x563da3620240 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31beffd640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f31a80a32c0 0x7f31a80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3615440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3615440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34432/0 (socket says 172.21.15.31:34432) 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 -- 172.21.15.31:0/1015347508 learned_addr learned my addr 172.21.15.31:0/1015347508 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3620240 0x563da3622630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bcff9640 1 -- 172.21.15.31:0/73085807 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f31b00cffe0 con 0x7f31c0101de0 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 -- 172.21.15.31:0/1015347508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3620240 msgr2=0x563da3622630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3620240 0x563da3622630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 -- 172.21.15.31:0/1015347508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 msgr2=0x563da361fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 -- 172.21.15.31:0/1015347508 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da3512d40 con 0x563da3615060 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31beffd640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f31a80a32c0 0x7f31a80a5780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f31c0075f80 tx=0x7f31b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.380+0000 7f31bffff640 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3615440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31b802f540 tx=0x7f31b8005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7649c3c47081e5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31a5ffb640 1 -- 172.21.15.31:0/1015347508 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8031c90 con 0x563da3615060 2024-09-16T00:47:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31a5ffb640 1 -- 172.21.15.31:0/1015347508 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b8036040 con 0x563da3615060 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31a5ffb640 1 -- 172.21.15.31:0/1015347508 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8004030 con 0x563da3615060 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31c7fe5740 1 -- 172.21.15.31:0/1015347508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 msgr2=0x563da3615440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3615440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31b802f540 tx=0x7f31b8005d80 comp rx=0 tx=0).stop 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31c7fe5740 1 -- 172.21.15.31:0/1015347508 shutdown_connections 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3620240 0x563da3622630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/1015347508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3615440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31c7fe5740 1 -- 172.21.15.31:0/1015347508 >> 172.21.15.31:0/1015347508 conn(0x563da36115f0 msgr2=0x563da36119d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31c7fe5740 1 -- 172.21.15.31:0/1015347508 shutdown_connections 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.381+0000 7f31c7fe5740 1 -- 172.21.15.31:0/1015347508 wait complete. 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31c7fe5740 1 -- start start 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31c7fe5740 1 Processor -- start 2024-09-16T00:47:23.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615060 0x563da361d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563da3622c90 con 0x563da3615980 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563da3502f10 con 0x563da3620240 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563da3622e10 con 0x563da3615060 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31c7fe5740 10 client.0 _ll_register_callbacks cb 0x563da3526db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.382+0000 7f31beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54562/0 (socket says 172.21.15.31:54562) 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31beffd640 1 -- 172.21.15.31:0/2409081903 learned_addr learned my addr 172.21.15.31:0/2409081903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615060 0x563da361d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31bffff640 1 -- 172.21.15.31:0/2409081903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615060 msgr2=0x563da361d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31bffff640 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615060 0x563da361d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31bffff640 1 -- 172.21.15.31:0/2409081903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 msgr2=0x563da361d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31bffff640 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31bffff640 1 -- 172.21.15.31:0/2409081903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da3289580 con 0x563da3620240 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615060 0x563da361d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31beffd640 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31bffff640 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361dbf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009ca0 tx=0x7f31b802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=164b255a65d6742e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f319effd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f319effd640 1 -- 172.21.15.31:0/2409081903 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b802fae0 con 0x563da3620240 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f31c7fe5740 1 -- 172.21.15.31:0/2409081903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31b8036040 con 0x563da3620240 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.383+0000 7f319effd640 1 -- 172.21.15.31:0/2409081903 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f31b802fc80 con 0x563da3620240 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f319effd640 1 -- 172.21.15.31:0/2409081903 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f31b8059050 con 0x563da3620240 2024-09-16T00:47:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f31c7fe5740 10 client.25823 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f31c7fe5740 1 -- 172.21.15.31:0/2409081903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563da3353960 con 0x563da3620240 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f319effd640 1 -- 172.21.15.31:0/2409081903 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b8036040 con 0x563da3620240 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f319effd640 1 -- 172.21.15.31:0/2409081903 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f31b804a600 con 0x563da3620240 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f31c7fe5740 10 client.25823 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f31c7fe5740 10 client.25823 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f31c7fe5740 20 client.25823 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563da362fab0 0x563da3631ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f31c7fe5740 4 client.25823 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f31c7fe5740 1 -- 172.21.15.31:0/2409081903 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563da3587b90 con 0x563da362fab0 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.384+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563da362fab0 0x563da3631ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.385+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563da362fab0 0x563da3631ea0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acdcb4c7e8515851 server_cookie=2c276161ea37fceb in_seq=0 out_seq=0 2024-09-16T00:47:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.385+0000 7f319effd640 10 client.25823 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.386+0000 7f319effd640 1 -- 172.21.15.31:0/2409081903 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563da3587b90 con 0x563da362fab0 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.386+0000 7f319effd640 10 client.25823 handle_command_reply: tid=0 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.386+0000 7f31c7fe5740 1 client.25823 shutdown 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.386+0000 7f31c7fe5740 1 -- 172.21.15.31:0/2409081903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 msgr2=0x563da361dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.386+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361dbf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009ca0 tx=0x7f31b802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.386+0000 7f31c7fe5740 1 -- 172.21.15.31:0/2409081903 shutdown_connections 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.386+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361dbf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.386+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615980 0x563da361d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.386+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563da362fab0 0x563da3631ea0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.387+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/2409081903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615060 0x563da361d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.387+0000 7f31c7fe5740 1 -- 172.21.15.31:0/2409081903 >> 172.21.15.31:0/2409081903 conn(0x563da36115f0 msgr2=0x563da3617320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:23.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.387+0000 7f319effd640 0 client.25823 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:23.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.387+0000 7f31c7fe5740 1 -- 172.21.15.31:0/2409081903 shutdown_connections 2024-09-16T00:47:23.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.387+0000 7f31c7fe5740 1 -- 172.21.15.31:0/2409081903 wait complete. 2024-09-16T00:47:23.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.387+0000 7f31c7fe5740 20 client.25823 trim_cache size 0 max 16384 2024-09-16T00:47:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31c7fe5740 1 Processor -- start 2024-09-16T00:47:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31c7fe5740 1 -- start start 2024-09-16T00:47:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3610640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3637530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3620240 0x563da3637a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f31c01aee40 con 0x563da3620240 2024-09-16T00:47:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f31c01b16c0 con 0x563da3615060 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31c01af4b0 con 0x563da3615980 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3620240 0x563da3637a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3620240 0x563da3637a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54572/0 (socket says 172.21.15.31:54572) 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3610640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bffff640 1 -- 172.21.15.31:0/926834105 learned_addr learned my addr 172.21.15.31:0/926834105 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31beffd640 1 --2- 172.21.15.31:0/926834105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3637530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bffff640 1 -- 172.21.15.31:0/926834105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 msgr2=0x563da3637530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bffff640 1 --2- 172.21.15.31:0/926834105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3637530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bffff640 1 -- 172.21.15.31:0/926834105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 msgr2=0x563da3610640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bffff640 1 --2- 172.21.15.31:0/926834105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3610640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bffff640 1 -- 172.21.15.31:0/926834105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da361c120 con 0x563da3620240 2024-09-16T00:47:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.407+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/926834105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3610640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.408+0000 7f31bffff640 1 --2- 172.21.15.31:0/926834105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3620240 0x563da3637a70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f31b8004af0 tx=0x7f31b8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1884ede31c5bd2d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.408+0000 7f31a57fa640 1 -- 172.21.15.31:0/926834105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b804a640 con 0x563da3620240 2024-09-16T00:47:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.408+0000 7f31a57fa640 1 -- 172.21.15.31:0/926834105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b8036040 con 0x563da3620240 2024-09-16T00:47:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.408+0000 7f31a57fa640 1 -- 172.21.15.31:0/926834105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8004040 con 0x563da3620240 2024-09-16T00:47:23.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 -- 172.21.15.31:0/926834105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3620240 msgr2=0x563da3637a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:23.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/926834105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3620240 0x563da3637a70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f31b8004af0 tx=0x7f31b8009360 comp rx=0 tx=0).stop 2024-09-16T00:47:23.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 -- 172.21.15.31:0/926834105 shutdown_connections 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/926834105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3620240 0x563da3637a70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/926834105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3637530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/926834105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3615060 0x563da3610640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 -- 172.21.15.31:0/926834105 >> 172.21.15.31:0/926834105 conn(0x563da36115f0 msgr2=0x563da360ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 -- 172.21.15.31:0/926834105 shutdown_connections 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 -- 172.21.15.31:0/926834105 wait complete. 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 -- start start 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.409+0000 7f31c7fe5740 1 Processor -- start 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615060 0x563da3628710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3619e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31c7fe5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f31c01afb80 con 0x563da3615060 2024-09-16T00:47:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f31c01b0ff0 con 0x563da3620240 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31c7fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31c01b0250 con 0x563da3615980 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31c7fe5740 10 client.0 _ll_register_callbacks cb 0x7f31b8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31beffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3619e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615060 0x563da3628710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31beffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3619e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:48732/0 (socket says 172.21.15.31:48732) 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34456/0 (socket says 172.21.15.31:34456) 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bffff640 1 -- 172.21.15.31:0/4146489676 learned_addr learned my addr 172.21.15.31:0/4146489676 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615060 0x563da3628710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54586/0 (socket says 172.21.15.31:54586) 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bffff640 1 -- 172.21.15.31:0/4146489676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 msgr2=0x563da3619e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bffff640 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3619e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bffff640 1 -- 172.21.15.31:0/4146489676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615060 msgr2=0x563da3628710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bffff640 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615060 0x563da3628710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bffff640 1 -- 172.21.15.31:0/4146489676 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da36157b0 con 0x563da3620240 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31beffd640 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3619e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.410+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615060 0x563da3628710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.411+0000 7f31bffff640 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361a3a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f31b8031c60 tx=0x7f31b802ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d5d1dc7232811f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.411+0000 7f319effd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.411+0000 7f319effd640 1 -- 172.21.15.31:0/4146489676 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b804a640 con 0x563da3620240 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.411+0000 7f319effd640 1 -- 172.21.15.31:0/4146489676 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f31b8003970 con 0x563da3620240 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.411+0000 7f31c7fe5740 1 -- 172.21.15.31:0/4146489676 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31b8036040 con 0x563da3620240 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.411+0000 7f319effd640 1 -- 172.21.15.31:0/4146489676 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f31b806d050 con 0x563da3620240 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.411+0000 7f31c7fe5740 10 client.25829 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.411+0000 7f31c7fe5740 1 -- 172.21.15.31:0/4146489676 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563da369a330 con 0x563da3620240 2024-09-16T00:47:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.411+0000 7f319effd640 1 -- 172.21.15.31:0/4146489676 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b8036040 con 0x563da3620240 2024-09-16T00:47:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f319effd640 1 -- 172.21.15.31:0/4146489676 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f31b8004540 con 0x563da3620240 2024-09-16T00:47:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f31c7fe5740 10 client.25829 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f31c7fe5740 10 client.25829 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:47:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f31c7fe5740 20 client.25829 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563da369a8e0 0x563da369ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f31c7fe5740 4 client.25829 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9da9a417", "release": true, "await": true}] 2024-09-16T00:47:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f31c7fe5740 1 -- 172.21.15.31:0/4146489676 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9da9a417", "release": true, "await": true}) -- 0x563da35820a0 con 0x563da369a8e0 2024-09-16T00:47:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563da369a8e0 0x563da369ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f319effd640 10 client.25829 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:23.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:23.412+0000 7f31bf7fe640 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563da369a8e0 0x563da369ccd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8329aead3e715717 server_cookie=81f1f5de60c3fd01 in_seq=0 out_seq=0 2024-09-16T00:47:24.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:23 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:47:24.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:23 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:47:24.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:23 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:47:24.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f319effd640 1 -- 172.21.15.31:0/4146489676 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563da35820a0 con 0x563da369a8e0 2024-09-16T00:47:24.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f319effd640 10 client.25829 handle_command_reply: tid=0 2024-09-16T00:47:24.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 client.25829 shutdown 2024-09-16T00:47:24.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 -- 172.21.15.31:0/4146489676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 msgr2=0x563da361a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:24.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361a3a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f31b8031c60 tx=0x7f31b802ff00 comp rx=0 tx=0).stop 2024-09-16T00:47:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 -- 172.21.15.31:0/4146489676 shutdown_connections 2024-09-16T00:47:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563da3620240 0x563da361a3a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x563da369a8e0 0x563da369ccd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563da3615980 0x563da3619e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 --2- 172.21.15.31:0/4146489676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563da3615060 0x563da3628710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f319effd640 0 client.25829 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 -- 172.21.15.31:0/4146489676 >> 172.21.15.31:0/4146489676 conn(0x563da36115f0 msgr2=0x563da361f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 -- 172.21.15.31:0/4146489676 shutdown_connections 2024-09-16T00:47:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 1 -- 172.21.15.31:0/4146489676 wait complete. 2024-09-16T00:47:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.458+0000 7f31c7fe5740 20 client.25829 trim_cache size 0 max 16384 2024-09-16T00:47:24.461 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:47:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:47:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:47:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 24, 2024-09-16T00:47:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": { 2024-09-16T00:47:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "9da9a417": { 2024-09-16T00:47:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "version": 24, 2024-09-16T00:47:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "age_ref": 0.0, 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "timeout": 90.0, 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "expiration": 145.3, 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "members": { 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "file:/": { 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "excluded": false, 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "state": { 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RELEASED", 2024-09-16T00:47:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0 2024-09-16T00:47:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:24.464 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:47:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 -- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f31a80a32c0 msgr2=0x7f31a80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f31a80a32c0 0x7f31a80a5780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f31c0075f80 tx=0x7f31b4005f90 comp rx=0 tx=0).stop 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 -- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0101de0 msgr2=0x7f31c00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0101de0 0x7f31c00744f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f31b000e9f0 tx=0x7f31b000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 -- 172.21.15.31:0/73085807 shutdown_connections 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f31a80a32c0 0x7f31a80a5780 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31c0106f10 0x7f31c0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31c0102790 0x7f31c0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 --2- 172.21.15.31:0/73085807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31c0101de0 0x7f31c00744f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.459+0000 7f31c5c0c640 1 -- 172.21.15.31:0/73085807 >> 172.21.15.31:0/73085807 conn(0x7f31c00fd8c0 msgr2=0x7f31c00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.460+0000 7f31c5c0c640 1 -- 172.21.15.31:0/73085807 shutdown_connections 2024-09-16T00:47:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:24.460+0000 7f31c5c0c640 1 -- 172.21.15.31:0/73085807 wait complete. 2024-09-16T00:47:25.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:24 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:47:25.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:24 smithi081 ceph-mon[35539]: pgmap v544: 129 pgs: 129 active+clean; 601 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 765 KiB/s wr, 1.40k op/s 2024-09-16T00:47:25.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:24 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:47:25.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:24 smithi102 ceph-mon[35364]: pgmap v544: 129 pgs: 129 active+clean; 601 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 765 KiB/s wr, 1.40k op/s 2024-09-16T00:47:25.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:24 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:47:25.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:24 smithi031 ceph-mon[30956]: pgmap v544: 129 pgs: 129 active+clean; 601 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 765 KiB/s wr, 1.40k op/s 2024-09-16T00:47:25.493 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9da9a417', total seconds elapsed: 30.8 2024-09-16T00:47:26.190 INFO:tasks.workunit.client.0.smithi031.stdout: 117 33350 25 21814 26 21623 73 2024-09-16T00:47:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:26 smithi081 ceph-mon[35539]: pgmap v545: 129 pgs: 129 active+clean; 601 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 453 KiB/s wr, 1.23k op/s 2024-09-16T00:47:27.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:26 smithi102 ceph-mon[35364]: pgmap v545: 129 pgs: 129 active+clean; 601 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 453 KiB/s wr, 1.23k op/s 2024-09-16T00:47:27.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:26 smithi031 ceph-mon[30956]: pgmap v545: 129 pgs: 129 active+clean; 601 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 453 KiB/s wr, 1.23k op/s 2024-09-16T00:47:29.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:28 smithi081 ceph-mon[35539]: pgmap v546: 129 pgs: 129 active+clean; 603 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 702 KiB/s wr, 1.27k op/s 2024-09-16T00:47:29.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:28 smithi102 ceph-mon[35364]: pgmap v546: 129 pgs: 129 active+clean; 603 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 702 KiB/s wr, 1.27k op/s 2024-09-16T00:47:29.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:28 smithi031 ceph-mon[30956]: pgmap v546: 129 pgs: 129 active+clean; 603 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 702 KiB/s wr, 1.27k op/s 2024-09-16T00:47:31.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:30 smithi031 ceph-mon[30956]: pgmap v547: 129 pgs: 129 active+clean; 604 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 707 KiB/s wr, 1.08k op/s 2024-09-16T00:47:31.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:30 smithi102 ceph-mon[35364]: pgmap v547: 129 pgs: 129 active+clean; 604 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 707 KiB/s wr, 1.08k op/s 2024-09-16T00:47:31.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:30 smithi081 ceph-mon[35539]: pgmap v547: 129 pgs: 129 active+clean; 604 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 707 KiB/s wr, 1.08k op/s 2024-09-16T00:47:33.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:33 smithi102 ceph-mon[35364]: pgmap v548: 129 pgs: 129 active+clean; 605 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 842 KiB/s wr, 921 op/s 2024-09-16T00:47:33.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:33 smithi031 ceph-mon[30956]: pgmap v548: 129 pgs: 129 active+clean; 605 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 842 KiB/s wr, 921 op/s 2024-09-16T00:47:33.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:33 smithi081 ceph-mon[35539]: pgmap v548: 129 pgs: 129 active+clean; 605 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 842 KiB/s wr, 921 op/s 2024-09-16T00:47:35.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:35 smithi102 ceph-mon[35364]: pgmap v549: 129 pgs: 129 active+clean; 614 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.3 MiB/s wr, 860 op/s 2024-09-16T00:47:35.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:35 smithi031 ceph-mon[30956]: pgmap v549: 129 pgs: 129 active+clean; 614 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.3 MiB/s wr, 860 op/s 2024-09-16T00:47:35.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:35 smithi081 ceph-mon[35539]: pgmap v549: 129 pgs: 129 active+clean; 614 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.3 MiB/s wr, 860 op/s 2024-09-16T00:47:36.190 INFO:tasks.workunit.client.0.smithi031.stdout: 124 1004928 244 655087 340 587492 959 2024-09-16T00:47:37.298 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:37 smithi081 ceph-mon[35539]: pgmap v550: 129 pgs: 129 active+clean; 613 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 467 op/s 2024-09-16T00:47:37.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:37 smithi102 ceph-mon[35364]: pgmap v550: 129 pgs: 129 active+clean; 613 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 467 op/s 2024-09-16T00:47:37.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:37 smithi031 ceph-mon[30956]: pgmap v550: 129 pgs: 129 active+clean; 613 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 467 op/s 2024-09-16T00:47:37.638 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:47:37.638 INFO:tasks.workunit.client.0.smithi031.stdout:Final score for writes: 124 2024-09-16T00:47:37.638 INFO:tasks.workunit.client.0.smithi031.stdout:Final score for reads : 404534 2024-09-16T00:47:37.639 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:47:37.639 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:47:37.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:47:39.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:39 smithi102 ceph-mon[35364]: pgmap v551: 129 pgs: 129 active+clean; 620 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 514 op/s 2024-09-16T00:47:39.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:39 smithi031 ceph-mon[30956]: pgmap v551: 129 pgs: 129 active+clean; 620 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 514 op/s 2024-09-16T00:47:39.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:39 smithi081 ceph-mon[35539]: pgmap v551: 129 pgs: 129 active+clean; 620 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 514 op/s 2024-09-16T00:47:40.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:47:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:41.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:41 smithi102 ceph-mon[35364]: pgmap v552: 129 pgs: 129 active+clean; 620 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 476 op/s 2024-09-16T00:47:41.305 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:41 smithi031 ceph-mon[30956]: pgmap v552: 129 pgs: 129 active+clean; 620 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 476 op/s 2024-09-16T00:47:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:41 smithi081 ceph-mon[35539]: pgmap v552: 129 pgs: 129 active+clean; 620 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 476 op/s 2024-09-16T00:47:41.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.847+0000 7f13990fa640 1 -- 172.21.15.31:0/3906265853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13941027c0 msgr2=0x7f1394106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:41.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.847+0000 7f13990fa640 1 --2- 172.21.15.31:0/3906265853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13941027c0 0x7f1394106810 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f137c009900 tx=0x7f137c02a410 comp rx=0 tx=0).stop 2024-09-16T00:47:41.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.847+0000 7f13990fa640 1 -- 172.21.15.31:0/3906265853 shutdown_connections 2024-09-16T00:47:41.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.847+0000 7f13990fa640 1 --2- 172.21.15.31:0/3906265853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1394106f40 0x7f139410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.847+0000 7f13990fa640 1 --2- 172.21.15.31:0/3906265853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13941027c0 0x7f1394106810 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.847+0000 7f13990fa640 1 --2- 172.21.15.31:0/3906265853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1394101e10 0x7f13941021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.847+0000 7f13990fa640 1 -- 172.21.15.31:0/3906265853 >> 172.21.15.31:0/3906265853 conn(0x7f13940fd8d0 msgr2=0x7f13940ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.847+0000 7f13990fa640 1 -- 172.21.15.31:0/3906265853 shutdown_connections 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.847+0000 7f13990fa640 1 -- 172.21.15.31:0/3906265853 wait complete. 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f13990fa640 1 Processor -- start 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f13990fa640 1 -- start start 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f13990fa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1394101e10 0x7f13941984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f13990fa640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13941027c0 0x7f1394198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f13990fa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1394106f40 0x7f139419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f13990fa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1394111fa0 con 0x7f1394106f40 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f13990fa640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1394111e20 con 0x7f1394101e10 2024-09-16T00:47:41.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f13990fa640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1394112120 con 0x7f13941027c0 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f139359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1394106f40 0x7f139419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f139359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1394106f40 0x7f139419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46314/0 (socket says 172.21.15.31:46314) 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.848+0000 7f139359e640 1 -- 172.21.15.31:0/314142598 learned_addr learned my addr 172.21.15.31:0/314142598 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f1392d9d640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1394101e10 0x7f13941984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f138bfff640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13941027c0 0x7f1394198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f139359e640 1 -- 172.21.15.31:0/314142598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13941027c0 msgr2=0x7f1394198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f139359e640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13941027c0 0x7f1394198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f139359e640 1 -- 172.21.15.31:0/314142598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1394101e10 msgr2=0x7f13941984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f139359e640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1394101e10 0x7f13941984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f139359e640 1 -- 172.21.15.31:0/314142598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139419d470 con 0x7f1394106f40 2024-09-16T00:47:41.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f1392d9d640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1394101e10 0x7f13941984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:41.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f139359e640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1394106f40 0x7f139419cd90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f138c00efd0 tx=0x7f138c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:41.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f1390d99640 1 -- 172.21.15.31:0/314142598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c003b70 con 0x7f1394106f40 2024-09-16T00:47:41.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f13990fa640 1 -- 172.21.15.31:0/314142598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139419d760 con 0x7f1394106f40 2024-09-16T00:47:41.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f1390d99640 1 -- 172.21.15.31:0/314142598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f138c003d10 con 0x7f1394106f40 2024-09-16T00:47:41.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f1390d99640 1 -- 172.21.15.31:0/314142598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c0109f0 con 0x7f1394106f40 2024-09-16T00:47:41.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.849+0000 7f13990fa640 1 -- 172.21.15.31:0/314142598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13941a5040 con 0x7f1394106f40 2024-09-16T00:47:41.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.852+0000 7f1390d99640 1 -- 172.21.15.31:0/314142598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f138c004020 con 0x7f1394106f40 2024-09-16T00:47:41.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.852+0000 7f139b4d3740 1 Processor -- start 2024-09-16T00:47:41.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.852+0000 7f139b4d3740 1 -- start start 2024-09-16T00:47:41.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.852+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 0x559c12d38880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.852+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d43140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d45a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f139b4d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559c12c9e2e0 con 0x559c12d384a0 2024-09-16T00:47:41.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f139b4d3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x559c12c45ce0 con 0x559c12d43680 2024-09-16T00:47:41.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f139b4d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559c12c9a2a0 con 0x559c12d38dc0 2024-09-16T00:47:41.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1392d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d43140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f138bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 0x559c12d38880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1392d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d43140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:45182/0 (socket says 172.21.15.31:45182) 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1392d9d640 1 -- 172.21.15.31:0/2216661173 learned_addr learned my addr 172.21.15.31:0/2216661173 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f138bfff640 1 --2- 172.21.15.31:0/2216661173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d45a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1390d99640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13800a3240 0x7f13800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1392d9d640 1 -- 172.21.15.31:0/2216661173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 msgr2=0x559c12d45a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1392d9d640 1 --2- 172.21.15.31:0/2216661173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d45a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1392d9d640 1 -- 172.21.15.31:0/2216661173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 msgr2=0x559c12d38880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1392d9d640 1 --2- 172.21.15.31:0/2216661173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 0x559c12d38880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1392d9d640 1 -- 172.21.15.31:0/2216661173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c12c2a440 con 0x559c12d38dc0 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f139359e640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13800a3240 0x7f13800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1390d99640 1 -- 172.21.15.31:0/314142598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f138c0d0060 con 0x7f1394106f40 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.853+0000 7f1392d9d640 1 --2- 172.21.15.31:0/2216661173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d43140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13840115f0 tx=0x7f1384011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f650dca094666f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:41.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.854+0000 7f13897fa640 1 -- 172.21.15.31:0/2216661173 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138401e070 con 0x559c12d38dc0 2024-09-16T00:47:41.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.854+0000 7f13897fa640 1 -- 172.21.15.31:0/2216661173 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f138400fb20 con 0x559c12d38dc0 2024-09-16T00:47:41.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.854+0000 7f13897fa640 1 -- 172.21.15.31:0/2216661173 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384019bd0 con 0x559c12d38dc0 2024-09-16T00:47:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.854+0000 7f139b4d3740 1 -- 172.21.15.31:0/2216661173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 msgr2=0x559c12d43140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.854+0000 7f139359e640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13800a3240 0x7f13800a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f138c098730 tx=0x7f138c066d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.854+0000 7f139b4d3740 1 --2- 172.21.15.31:0/2216661173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d43140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13840115f0 tx=0x7f1384011ab0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.854+0000 7f139b4d3740 1 -- 172.21.15.31:0/2216661173 shutdown_connections 2024-09-16T00:47:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 --2- 172.21.15.31:0/2216661173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d45a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 --2- 172.21.15.31:0/2216661173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d43140 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 --2- 172.21.15.31:0/2216661173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 0x559c12d38880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 -- 172.21.15.31:0/2216661173 >> 172.21.15.31:0/2216661173 conn(0x559c12d34a30 msgr2=0x559c12d34e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:41.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 -- 172.21.15.31:0/2216661173 shutdown_connections 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 -- 172.21.15.31:0/2216661173 wait complete. 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 -- start start 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 Processor -- start 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d384a0 0x559c12d42ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d38dc0 0x559c12d402a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d40800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559c12d46250 con 0x559c12d38dc0 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x559c12c59050 con 0x559c12d43680 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559c12d463d0 con 0x559c12d384a0 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f139b4d3740 10 client.0 _ll_register_callbacks cb 0x559c12c99160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f1392d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d38dc0 0x559c12d402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f138bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d384a0 0x559c12d42ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f1392d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d38dc0 0x559c12d402a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46338/0 (socket says 172.21.15.31:46338) 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f1392d9d640 1 -- 172.21.15.31:0/1192002294 learned_addr learned my addr 172.21.15.31:0/1192002294 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.855+0000 7f138bfff640 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d40800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f1392d9d640 1 -- 172.21.15.31:0/1192002294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d384a0 msgr2=0x559c12d42ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f1392d9d640 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d384a0 0x559c12d42ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f1392d9d640 1 -- 172.21.15.31:0/1192002294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 msgr2=0x559c12d40800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f1392d9d640 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d40800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f1392d9d640 1 -- 172.21.15.31:0/1192002294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c12a0f3c0 con 0x559c12d38dc0 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f1392d9d640 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d38dc0 0x559c12d402a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f13840198b0 tx=0x7f138400f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809cc064e3c55a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f137a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f137a7fc640 1 -- 172.21.15.31:0/1192002294 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138401e040 con 0x559c12d38dc0 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f139b4d3740 1 -- 172.21.15.31:0/1192002294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f138400fb20 con 0x559c12d38dc0 2024-09-16T00:47:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f137a7fc640 1 -- 172.21.15.31:0/1192002294 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f1384004040 con 0x559c12d38dc0 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f137a7fc640 1 -- 172.21.15.31:0/1192002294 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f138402f050 con 0x559c12d38dc0 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f139b4d3740 10 client.16167 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.856+0000 7f139b4d3740 1 -- 172.21.15.31:0/1192002294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559c12d51c20 con 0x559c12d38dc0 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f137a7fc640 1 -- 172.21.15.31:0/1192002294 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f138400fb20 con 0x559c12d38dc0 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f137a7fc640 1 -- 172.21.15.31:0/1192002294 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f13840035f0 con 0x559c12d38dc0 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f139b4d3740 10 client.16167 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f139b4d3740 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f139b4d3740 20 client.16167 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x559c12d53030 0x559c12d55420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f139b4d3740 4 client.16167 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f139b4d3740 1 -- 172.21.15.31:0/1192002294 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c12cab850 con 0x559c12d53030 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f138bfff640 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x559c12d53030 0x559c12d55420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f137a7fc640 10 client.16167 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.857+0000 7f138bfff640 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x559c12d53030 0x559c12d55420 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31dcbf5bdfbc1b0a server_cookie=adaf5df0f1c63a03 in_seq=0 out_seq=0 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f137a7fc640 1 -- 172.21.15.31:0/1192002294 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c12cab850 con 0x559c12d53030 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f137a7fc640 10 client.16167 handle_command_reply: tid=0 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 client.16167 shutdown 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 -- 172.21.15.31:0/1192002294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d38dc0 msgr2=0x559c12d402a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d38dc0 0x559c12d402a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f13840198b0 tx=0x7f138400f870 comp rx=0 tx=0).stop 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 -- 172.21.15.31:0/1192002294 shutdown_connections 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d40800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d38dc0 0x559c12d402a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x559c12d53030 0x559c12d55420 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1192002294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d384a0 0x559c12d42ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f137a7fc640 0 client.16167 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 -- 172.21.15.31:0/1192002294 >> 172.21.15.31:0/1192002294 conn(0x559c12d34a30 msgr2=0x559c12d33b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:41.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 -- 172.21.15.31:0/1192002294 shutdown_connections 2024-09-16T00:47:41.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 1 -- 172.21.15.31:0/1192002294 wait complete. 2024-09-16T00:47:41.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.859+0000 7f139b4d3740 20 client.16167 trim_cache size 0 max 16384 2024-09-16T00:47:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.877+0000 7f139b4d3740 1 Processor -- start 2024-09-16T00:47:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f139b4d3740 1 -- start start 2024-09-16T00:47:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 0x559c12d4f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d5ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d5b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f139b4d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13941aaa70 con 0x559c12d384a0 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f139b4d3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f13941ad2f0 con 0x559c12d43680 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f139b4d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13941ab0e0 con 0x559c12d38dc0 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f138bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 0x559c12d4f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f138bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 0x559c12d4f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46352/0 (socket says 172.21.15.31:46352) 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f138bfff640 1 -- 172.21.15.31:0/2465896967 learned_addr learned my addr 172.21.15.31:0/2465896967 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f1392d9d640 1 --2- 172.21.15.31:0/2465896967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d5ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.878+0000 7f138bfff640 1 --2- 172.21.15.31:0/2465896967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d5b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.879+0000 7f1392d9d640 1 -- 172.21.15.31:0/2465896967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 msgr2=0x559c12d5b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.879+0000 7f1392d9d640 1 --2- 172.21.15.31:0/2465896967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d5b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.879+0000 7f1392d9d640 1 -- 172.21.15.31:0/2465896967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 msgr2=0x559c12d4f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.879+0000 7f1392d9d640 1 --2- 172.21.15.31:0/2465896967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 0x559c12d4f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.879+0000 7f1392d9d640 1 -- 172.21.15.31:0/2465896967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c12d40b30 con 0x559c12d38dc0 2024-09-16T00:47:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.879+0000 7f1392d9d640 1 --2- 172.21.15.31:0/2465896967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d5ac00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1384003b60 tx=0x7f1384005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a202eaaa562f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.879+0000 7f1388ff9640 1 -- 172.21.15.31:0/2465896967 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138400fb20 con 0x559c12d38dc0 2024-09-16T00:47:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.879+0000 7f1388ff9640 1 -- 172.21.15.31:0/2465896967 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1384019dc0 con 0x559c12d38dc0 2024-09-16T00:47:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.879+0000 7f1388ff9640 1 -- 172.21.15.31:0/2465896967 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384005350 con 0x559c12d38dc0 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 -- 172.21.15.31:0/2465896967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 msgr2=0x559c12d5ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 --2- 172.21.15.31:0/2465896967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d5ac00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1384003b60 tx=0x7f1384005de0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 -- 172.21.15.31:0/2465896967 shutdown_connections 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 --2- 172.21.15.31:0/2465896967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d43680 0x559c12d5b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 --2- 172.21.15.31:0/2465896967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d5ac00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 --2- 172.21.15.31:0/2465896967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d384a0 0x559c12d4f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 -- 172.21.15.31:0/2465896967 >> 172.21.15.31:0/2465896967 conn(0x559c12d34a30 msgr2=0x559c12d3b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 -- 172.21.15.31:0/2465896967 shutdown_connections 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 -- 172.21.15.31:0/2465896967 wait complete. 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 -- start start 2024-09-16T00:47:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.880+0000 7f139b4d3740 1 Processor -- start 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d384a0 0x559c12d4bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d46df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f139b4d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d43680 0x559c12d47330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f139b4d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13941ab7b0 con 0x559c12d43680 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f139b4d3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f13941acc20 con 0x559c12d384a0 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f139b4d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13941abe80 con 0x559c12d38dc0 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f139b4d3740 10 client.0 _ll_register_callbacks cb 0x7f138401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f1392d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d46df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f138bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d43680 0x559c12d47330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f1392d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d46df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:45210/0 (socket says 172.21.15.31:45210) 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f1392d9d640 1 -- 172.21.15.31:0/1688142344 learned_addr learned my addr 172.21.15.31:0/1688142344 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f138bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d43680 0x559c12d47330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46358/0 (socket says 172.21.15.31:46358) 2024-09-16T00:47:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f138bfff640 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d384a0 0x559c12d4bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f1392d9d640 1 -- 172.21.15.31:0/1688142344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d384a0 msgr2=0x559c12d4bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f1392d9d640 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d384a0 0x559c12d4bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f1392d9d640 1 -- 172.21.15.31:0/1688142344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d43680 msgr2=0x559c12d47330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f1392d9d640 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d43680 0x559c12d47330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f1392d9d640 1 -- 172.21.15.31:0/1688142344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c12d47870 con 0x559c12d38dc0 2024-09-16T00:47:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f138bfff640 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d43680 0x559c12d47330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f1392d9d640 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d46df0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1384019800 tx=0x7f1384009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8958db4c2c10dca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.881+0000 7f137a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f137a7fc640 1 -- 172.21.15.31:0/1688142344 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840035f0 con 0x559c12d38dc0 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f137a7fc640 1 -- 172.21.15.31:0/1688142344 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f1384003be0 con 0x559c12d38dc0 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f139b4d3740 1 -- 172.21.15.31:0/1688142344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1384019dc0 con 0x559c12d38dc0 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f137a7fc640 1 -- 172.21.15.31:0/1688142344 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1384048050 con 0x559c12d38dc0 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f139b4d3740 10 client.25774 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f139b4d3740 1 -- 172.21.15.31:0/1688142344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559c12dc6140 con 0x559c12d38dc0 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f137a7fc640 1 -- 172.21.15.31:0/1688142344 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1384019dc0 con 0x559c12d38dc0 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f137a7fc640 1 -- 172.21.15.31:0/1688142344 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f13840054e0 con 0x559c12d38dc0 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f139b4d3740 10 client.25774 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f139b4d3740 10 client.25774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f139b4d3740 20 client.25774 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x559c12d53030 0x559c12dc87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:41.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f139b4d3740 4 client.25774 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:47:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.882+0000 7f139b4d3740 1 -- 172.21.15.31:0/1688142344 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c12ca5d60 con 0x559c12d53030 2024-09-16T00:47:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.883+0000 7f138bfff640 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x559c12d53030 0x559c12dc87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.883+0000 7f138bfff640 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x559c12d53030 0x559c12dc87e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06af26f98fbd478 server_cookie=da270261858204f1 in_seq=0 out_seq=0 2024-09-16T00:47:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.883+0000 7f137a7fc640 10 client.25774 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f137a7fc640 1 -- 172.21.15.31:0/1688142344 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559c12ca5d60 con 0x559c12d53030 2024-09-16T00:47:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f137a7fc640 10 client.25774 handle_command_reply: tid=0 2024-09-16T00:47:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f139b4d3740 1 client.25774 shutdown 2024-09-16T00:47:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f139b4d3740 1 -- 172.21.15.31:0/1688142344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 msgr2=0x559c12d46df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d46df0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1384019800 tx=0x7f1384009f60 comp rx=0 tx=0).stop 2024-09-16T00:47:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f139b4d3740 1 -- 172.21.15.31:0/1688142344 shutdown_connections 2024-09-16T00:47:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c12d43680 0x559c12d47330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x559c12d53030 0x559c12dc87e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c12d38dc0 0x559c12d46df0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f139b4d3740 1 --2- 172.21.15.31:0/1688142344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559c12d384a0 0x559c12d4bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f139b4d3740 1 -- 172.21.15.31:0/1688142344 >> 172.21.15.31:0/1688142344 conn(0x559c12d34a30 msgr2=0x559c12d33350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.885+0000 7f137a7fc640 0 client.25774 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f139b4d3740 1 -- 172.21.15.31:0/1688142344 shutdown_connections 2024-09-16T00:47:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f139b4d3740 1 -- 172.21.15.31:0/1688142344 wait complete. 2024-09-16T00:47:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f139b4d3740 20 client.25774 trim_cache size 0 max 16384 2024-09-16T00:47:41.890 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:47:41.890 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:47:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:47:41.891 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:47:41.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 -- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13800a3240 msgr2=0x7f13800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:41.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13800a3240 0x7f13800a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f138c098730 tx=0x7f138c066d10 comp rx=0 tx=0).stop 2024-09-16T00:47:41.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 -- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1394106f40 msgr2=0x7f139419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1394106f40 0x7f139419cd90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f138c00efd0 tx=0x7f138c00c6a0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 -- 172.21.15.31:0/314142598 shutdown_connections 2024-09-16T00:47:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1394106f40 0x7f139419cd90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f13800a3240 0x7f13800a5700 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13941027c0 0x7f1394198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 --2- 172.21.15.31:0/314142598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1394101e10 0x7f13941984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 -- 172.21.15.31:0/314142598 >> 172.21.15.31:0/314142598 conn(0x7f13940fd8d0 msgr2=0x7f13940fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 -- 172.21.15.31:0/314142598 shutdown_connections 2024-09-16T00:47:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:41.886+0000 7f13990fa640 1 -- 172.21.15.31:0/314142598 wait complete. 2024-09-16T00:47:42.700 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:47:42.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:47:43.049 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:43.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:43 smithi102 ceph-mon[35364]: pgmap v553: 129 pgs: 129 active+clean; 622 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 576 op/s 2024-09-16T00:47:43.347 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:43 smithi031 ceph-mon[30956]: pgmap v553: 129 pgs: 129 active+clean; 622 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 576 op/s 2024-09-16T00:47:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:43 smithi081 ceph-mon[35539]: pgmap v553: 129 pgs: 129 active+clean; 622 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 576 op/s 2024-09-16T00:47:44.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.158+0000 7fbc4741b640 1 -- 172.21.15.31:0/2962416090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc4010c160 msgr2=0x7fbc4010e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.158+0000 7fbc4741b640 1 --2- 172.21.15.31:0/2962416090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc4010c160 0x7fbc4010e5e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009ca0 tx=0x7fbc3802a950 comp rx=0 tx=0).stop 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.158+0000 7fbc4741b640 1 -- 172.21.15.31:0/2962416090 shutdown_connections 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.158+0000 7fbc4741b640 1 --2- 172.21.15.31:0/2962416090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc4010c160 0x7fbc4010e5e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.158+0000 7fbc4741b640 1 --2- 172.21.15.31:0/2962416090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc401036a0 0x7fbc4010bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.158+0000 7fbc4741b640 1 --2- 172.21.15.31:0/2962416090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc40102d80 0x7fbc40103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.158+0000 7fbc4741b640 1 -- 172.21.15.31:0/2962416090 >> 172.21.15.31:0/2962416090 conn(0x7fbc400fc780 msgr2=0x7fbc400feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.159+0000 7fbc4741b640 1 -- 172.21.15.31:0/2962416090 shutdown_connections 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.159+0000 7fbc4741b640 1 -- 172.21.15.31:0/2962416090 wait complete. 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.159+0000 7fbc4741b640 1 Processor -- start 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.159+0000 7fbc4741b640 1 -- start start 2024-09-16T00:47:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.159+0000 7fbc4741b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc40102d80 0x7fbc40198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.159+0000 7fbc4741b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc401036a0 0x7fbc401989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc4741b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4010c160 0x7fbc4019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc4741b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40111f00 con 0x7fbc401036a0 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc4741b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40111d80 con 0x7fbc40102d80 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc4741b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40112080 con 0x7fbc4010c160 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc449b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc401036a0 0x7fbc401989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc451b7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc40102d80 0x7fbc40198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc451b7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc40102d80 0x7fbc40198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:42024/0 (socket says 172.21.15.31:42024) 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc451b7640 1 -- 172.21.15.31:0/1249247163 learned_addr learned my addr 172.21.15.31:0/1249247163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc459b8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4010c160 0x7fbc4019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc449b6640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc401036a0 0x7fbc401989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46378/0 (socket says 172.21.15.31:46378) 2024-09-16T00:47:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc449b6640 1 -- 172.21.15.31:0/1249247163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4010c160 msgr2=0x7fbc4019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc449b6640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4010c160 0x7fbc4019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc449b6640 1 -- 172.21.15.31:0/1249247163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc40102d80 msgr2=0x7fbc40198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc449b6640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc40102d80 0x7fbc40198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc449b6640 1 -- 172.21.15.31:0/1249247163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4019d420 con 0x7fbc401036a0 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc449b6640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc401036a0 0x7fbc401989b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000e9f0 tx=0x7fbc3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc367fc640 1 -- 172.21.15.31:0/1249247163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000ccc0 con 0x7fbc401036a0 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.160+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4019d710 con 0x7fbc401036a0 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.161+0000 7fbc367fc640 1 -- 172.21.15.31:0/1249247163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc30010910 con 0x7fbc401036a0 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.161+0000 7fbc367fc640 1 -- 172.21.15.31:0/1249247163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30018960 con 0x7fbc401036a0 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.161+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc401a4ff0 con 0x7fbc401036a0 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.161+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc40103160 con 0x7fbc401036a0 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.163+0000 7fbc367fc640 1 -- 172.21.15.31:0/1249247163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbc30010ac0 con 0x7fbc401036a0 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.164+0000 7fbc367fc640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc2c0a32c0 0x7fbc2c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.164+0000 7fbc451b7640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc2c0a32c0 0x7fbc2c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.164+0000 7fbc367fc640 1 -- 172.21.15.31:0/1249247163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fbc300d03d0 con 0x7fbc401036a0 2024-09-16T00:47:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.164+0000 7fbc451b7640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc2c0a32c0 0x7fbc2c0a5780 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0095f0 tx=0x7fbc3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.167+0000 7fbc367fc640 1 -- 172.21.15.31:0/1249247163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc3009c6f0 con 0x7fbc401036a0 2024-09-16T00:47:44.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.385+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc40107200 con 0x7fbc401036a0 2024-09-16T00:47:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.386+0000 7fbc367fc640 1 -- 172.21.15.31:0/1249247163 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fbc300a09f0 con 0x7fbc401036a0 2024-09-16T00:47:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:47:44.386 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:47:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:47:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc2c0a32c0 msgr2=0x7fbc2c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc2c0a32c0 0x7fbc2c0a5780 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0095f0 tx=0x7fbc3c005f90 comp rx=0 tx=0).stop 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc401036a0 msgr2=0x7fbc401989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc401036a0 0x7fbc401989b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000e9f0 tx=0x7fbc3000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 shutdown_connections 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbc2c0a32c0 0x7fbc2c0a5780 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc4010c160 0x7fbc4019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc401036a0 0x7fbc401989b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 --2- 172.21.15.31:0/1249247163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc40102d80 0x7fbc40198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 >> 172.21.15.31:0/1249247163 conn(0x7fbc400fc780 msgr2=0x7fbc400fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 shutdown_connections 2024-09-16T00:47:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:44.388+0000 7fbc4741b640 1 -- 172.21.15.31:0/1249247163 wait complete. 2024-09-16T00:47:45.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:47:45.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:45 smithi102 ceph-mon[35364]: pgmap v554: 129 pgs: 129 active+clean; 610 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 822 op/s 2024-09-16T00:47:45.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:45 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1249247163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:47:45.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:45 smithi031 ceph-mon[30956]: pgmap v554: 129 pgs: 129 active+clean; 610 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 822 op/s 2024-09-16T00:47:45.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:45 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1249247163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:47:45.407 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:45.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:45 smithi081 ceph-mon[35539]: pgmap v554: 129 pgs: 129 active+clean; 610 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 822 op/s 2024-09-16T00:47:45.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:45 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1249247163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:47:46.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.132+0000 7f90ae567640 1 -- 172.21.15.31:0/1497359944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a81036e0 msgr2=0x7f90a8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:46.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.132+0000 7f90ae567640 1 --2- 172.21.15.31:0/1497359944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a81036e0 0x7f90a8107b90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f909c00fb40 tx=0x7f909c0329c0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.135+0000 7f90ae567640 1 -- 172.21.15.31:0/1497359944 shutdown_connections 2024-09-16T00:47:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.135+0000 7f90ae567640 1 --2- 172.21.15.31:0/1497359944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90a81082a0 0x7f90a810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.135+0000 7f90ae567640 1 --2- 172.21.15.31:0/1497359944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a81036e0 0x7f90a8107b90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.135+0000 7f90ae567640 1 --2- 172.21.15.31:0/1497359944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a8102d30 0x7f90a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.135+0000 7f90ae567640 1 -- 172.21.15.31:0/1497359944 >> 172.21.15.31:0/1497359944 conn(0x7f90a80fc7d0 msgr2=0x7f90a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:46.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.137+0000 7f90ae567640 1 -- 172.21.15.31:0/1497359944 shutdown_connections 2024-09-16T00:47:46.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.137+0000 7f90ae567640 1 -- 172.21.15.31:0/1497359944 wait complete. 2024-09-16T00:47:46.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.138+0000 7f90ae567640 1 Processor -- start 2024-09-16T00:47:46.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.138+0000 7f90ae567640 1 -- start start 2024-09-16T00:47:46.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.139+0000 7f90ae567640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a8102d30 0x7f90a8196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.139+0000 7f90ae567640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a81036e0 0x7f90a8196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.139+0000 7f90ae567640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90a81082a0 0x7f90a819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.139+0000 7f90ae567640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f90a810fed0 con 0x7f90a81036e0 2024-09-16T00:47:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.139+0000 7f90acb04640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90a81082a0 0x7f90a819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90acb04640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90a81082a0 0x7f90a819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:42044/0 (socket says 172.21.15.31:42044) 2024-09-16T00:47:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90acb04640 1 -- 172.21.15.31:0/2946355490 learned_addr learned my addr 172.21.15.31:0/2946355490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f90a810fd50 con 0x7f90a81082a0 2024-09-16T00:47:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8110050 con 0x7f90a8102d30 2024-09-16T00:47:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90a7fff640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a8102d30 0x7f90a8196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90acb04640 1 -- 172.21.15.31:0/2946355490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a8102d30 msgr2=0x7f90a8196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90acb04640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a8102d30 0x7f90a8196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90acb04640 1 -- 172.21.15.31:0/2946355490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a81036e0 msgr2=0x7f90a8196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90acb04640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a81036e0 0x7f90a8196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.140+0000 7f90acb04640 1 -- 172.21.15.31:0/2946355490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a819b3d0 con 0x7f90a81082a0 2024-09-16T00:47:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.141+0000 7f90a7fff640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a8102d30 0x7f90a8196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:47:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.144+0000 7f90acb04640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90a81082a0 0x7f90a819acf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f90a000b990 tx=0x7f90a000be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.148+0000 7f90a57fa640 1 -- 172.21.15.31:0/2946355490 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0019070 con 0x7f90a81082a0 2024-09-16T00:47:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.148+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a819b6c0 con 0x7f90a81082a0 2024-09-16T00:47:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.148+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a81a2fa0 con 0x7f90a81082a0 2024-09-16T00:47:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.150+0000 7f90a57fa640 1 -- 172.21.15.31:0/2946355490 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a000cba0 con 0x7f90a81082a0 2024-09-16T00:47:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.150+0000 7f90a57fa640 1 -- 172.21.15.31:0/2946355490 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0007650 con 0x7f90a81082a0 2024-09-16T00:47:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.153+0000 7f90b0940740 1 Processor -- start 2024-09-16T00:47:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.157+0000 7f90a57fa640 1 -- 172.21.15.31:0/2946355490 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f90a001c070 con 0x7f90a81082a0 2024-09-16T00:47:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.157+0000 7f90b0940740 1 -- start start 2024-09-16T00:47:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90b0940740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb30060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90b0940740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90b0940740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb3d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90b0940740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555c6ea962f0 con 0x555c6eb31b10 2024-09-16T00:47:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90b0940740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555c6ea3dcf0 con 0x555c6eb3b340 2024-09-16T00:47:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90b0940740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555c6ea922b0 con 0x555c6eb305a0 2024-09-16T00:47:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90a7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90a7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:45256/0 (socket says 172.21.15.31:45256) 2024-09-16T00:47:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90a7fff640 1 -- 172.21.15.31:0/1680839636 learned_addr learned my addr 172.21.15.31:0/1680839636 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90a57fa640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90900a3390 0x7f90900a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.158+0000 7f90a77fe640 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb3d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.159+0000 7f90acb04640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90900a3390 0x7f90900a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.159+0000 7f90a57fa640 1 -- 172.21.15.31:0/2946355490 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f90a00cf1e0 con 0x7f90a81082a0 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.159+0000 7f90a77fe640 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb30060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.159+0000 7f90a7fff640 1 -- 172.21.15.31:0/1680839636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 msgr2=0x555c6eb3d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.159+0000 7f90a7fff640 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb3d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.159+0000 7f90acb04640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90900a3390 0x7f90900a5850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f90a009b380 tx=0x7f90a0097040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.159+0000 7f90a7fff640 1 -- 172.21.15.31:0/1680839636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 msgr2=0x555c6eb30060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.159+0000 7f90a7fff640 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb30060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.160+0000 7f90a7fff640 1 -- 172.21.15.31:0/1680839636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c6ea223a0 con 0x555c6eb305a0 2024-09-16T00:47:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.160+0000 7f90a77fe640 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb30060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:47:46.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.160+0000 7f90a7fff640 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb309e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9098009c30 tx=0x7f9098010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=795084cec53e1585 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:46.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.161+0000 7f908e7fc640 1 -- 172.21.15.31:0/1680839636 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098004030 con 0x555c6eb305a0 2024-09-16T00:47:46.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.161+0000 7f908e7fc640 1 -- 172.21.15.31:0/1680839636 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90980041d0 con 0x555c6eb305a0 2024-09-16T00:47:46.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.162+0000 7f90b0940740 1 -- 172.21.15.31:0/1680839636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 msgr2=0x555c6eb309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:46.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.162+0000 7f90b0940740 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb309e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9098009c30 tx=0x7f9098010740 comp rx=0 tx=0).stop 2024-09-16T00:47:46.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.162+0000 7f90b0940740 1 -- 172.21.15.31:0/1680839636 shutdown_connections 2024-09-16T00:47:46.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.162+0000 7f90b0940740 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb3d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.162+0000 7f90b0940740 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb309e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.162+0000 7f90b0940740 1 --2- 172.21.15.31:0/1680839636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb30060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.162+0000 7f90b0940740 1 -- 172.21.15.31:0/1680839636 >> 172.21.15.31:0/1680839636 conn(0x555c6eaf9310 msgr2=0x555c6eb2cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:46.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.166+0000 7f90b0940740 1 -- 172.21.15.31:0/1680839636 shutdown_connections 2024-09-16T00:47:46.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.166+0000 7f90b0940740 1 -- 172.21.15.31:0/1680839636 wait complete. 2024-09-16T00:47:46.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.166+0000 7f90b0940740 1 -- start start 2024-09-16T00:47:46.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.167+0000 7f90b0940740 1 Processor -- start 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.167+0000 7f90b0940740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb305a0 0x555c6eb3ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.167+0000 7f90b0940740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb37eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.167+0000 7f90b0940740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb3b340 0x555c6eb383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90b0940740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555c6eb3e090 con 0x555c6eb31b10 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90b0940740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555c6ea51060 con 0x555c6eb305a0 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90b0940740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555c6eb3e210 con 0x555c6eb3b340 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90a77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb305a0 0x555c6eb3ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90a77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb3b340 0x555c6eb383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90a77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb305a0 0x555c6eb3ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:42062/0 (socket says 172.21.15.31:42062) 2024-09-16T00:47:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90a77fe640 1 -- 172.21.15.31:0/2591479604 learned_addr learned my addr 172.21.15.31:0/2591479604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:46.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90b0940740 10 client.0 _ll_register_callbacks cb 0x555c6e7a48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:46.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90a7fff640 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb37eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90a77fe640 1 -- 172.21.15.31:0/2591479604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb3b340 msgr2=0x555c6eb383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:46.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90a77fe640 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb3b340 0x555c6eb383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.168+0000 7f90a77fe640 1 -- 172.21.15.31:0/2591479604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 msgr2=0x555c6eb37eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:46.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.169+0000 7f90a77fe640 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb37eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.169+0000 7f90a77fe640 1 -- 172.21.15.31:0/2591479604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c6e8073c0 con 0x555c6eb305a0 2024-09-16T00:47:46.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.169+0000 7f90a77fe640 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb305a0 0x555c6eb3ac00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f909c005ab0 tx=0x7f909c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bff0482170880c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:46.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.169+0000 7f90877fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.169+0000 7f90877fe640 1 -- 172.21.15.31:0/2591479604 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c03b030 con 0x555c6eb305a0 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.169+0000 7f90b0940740 1 -- 172.21.15.31:0/2591479604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90980041d0 con 0x555c6eb305a0 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.169+0000 7f90877fe640 1 -- 172.21.15.31:0/2591479604 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f909c03c070 con 0x555c6eb305a0 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90877fe640 1 -- 172.21.15.31:0/2591479604 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f909c03b030 con 0x555c6eb305a0 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90b0940740 10 client.25862 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90b0940740 1 -- 172.21.15.31:0/2591479604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555c6eb49ac0 con 0x555c6eb305a0 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90877fe640 1 -- 172.21.15.31:0/2591479604 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f909c056070 con 0x555c6eb305a0 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90877fe640 1 -- 172.21.15.31:0/2591479604 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f909c041570 con 0x555c6eb305a0 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90b0940740 10 client.25862 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90b0940740 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90b0940740 20 client.25862 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90b0940740 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555c6eb4aed0 0x555c6eb4d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90b0940740 4 client.25862 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:47:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.170+0000 7f90b0940740 1 -- 172.21.15.31:0/2591479604 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c6eaa3860 con 0x555c6eb4aed0 2024-09-16T00:47:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.172+0000 7f90a7fff640 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555c6eb4aed0 0x555c6eb4d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.173+0000 7f90a7fff640 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555c6eb4aed0 0x555c6eb4d320 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3551d21ec82d53f4 server_cookie=ce5e727dfbe6f698 in_seq=0 out_seq=0 2024-09-16T00:47:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.173+0000 7f90877fe640 10 client.25862 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.174+0000 7f90877fe640 1 -- 172.21.15.31:0/2591479604 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c6eaa3860 con 0x555c6eb4aed0 2024-09-16T00:47:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.174+0000 7f90877fe640 10 client.25862 handle_command_reply: tid=0 2024-09-16T00:47:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.174+0000 7f90b0940740 1 client.25862 shutdown 2024-09-16T00:47:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.174+0000 7f90b0940740 1 -- 172.21.15.31:0/2591479604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb305a0 msgr2=0x555c6eb3ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.174+0000 7f90b0940740 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb305a0 0x555c6eb3ac00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f909c005ab0 tx=0x7f909c005fd0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90b0940740 1 -- 172.21.15.31:0/2591479604 shutdown_connections 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90b0940740 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555c6eb4aed0 0x555c6eb4d320 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90b0940740 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb3b340 0x555c6eb383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90b0940740 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb37eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90877fe640 0 client.25862 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90b0940740 1 --2- 172.21.15.31:0/2591479604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb305a0 0x555c6eb3ac00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90b0940740 1 -- 172.21.15.31:0/2591479604 >> 172.21.15.31:0/2591479604 conn(0x555c6eaf9310 msgr2=0x555c6eb32900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90b0940740 1 -- 172.21.15.31:0/2591479604 shutdown_connections 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90b0940740 1 -- 172.21.15.31:0/2591479604 wait complete. 2024-09-16T00:47:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.175+0000 7f90b0940740 20 client.25862 trim_cache size 0 max 16384 2024-09-16T00:47:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90b0940740 1 Processor -- start 2024-09-16T00:47:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90b0940740 1 -- start start 2024-09-16T00:47:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90b0940740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb47410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90b0940740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb3c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb3c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb47410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb3c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46442/0 (socket says 172.21.15.31:46442) 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a7fff640 1 -- 172.21.15.31:0/507791528 learned_addr learned my addr 172.21.15.31:0/507791528 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90b0940740 1 --2- 172.21.15.31:0/507791528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb3cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90b0940740 1 -- 172.21.15.31:0/507791528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f909400dc90 con 0x555c6eb31b10 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90b0940740 1 -- 172.21.15.31:0/507791528 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9094010570 con 0x555c6eb3b340 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90b0940740 1 -- 172.21.15.31:0/507791528 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f909400e360 con 0x555c6eb305a0 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a77fe640 1 --2- 172.21.15.31:0/507791528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb3cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a7fff640 1 -- 172.21.15.31:0/507791528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 msgr2=0x555c6eb47410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a7fff640 1 --2- 172.21.15.31:0/507791528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb47410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a7fff640 1 -- 172.21.15.31:0/507791528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 msgr2=0x555c6eb3cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a7fff640 1 --2- 172.21.15.31:0/507791528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb3cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.189+0000 7f90a7fff640 1 -- 172.21.15.31:0/507791528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c6eb38690 con 0x555c6eb31b10 2024-09-16T00:47:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.190+0000 7f90a7fff640 1 --2- 172.21.15.31:0/507791528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb3c730 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f90980041a0 tx=0x7f909801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4829eb4451bb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.190+0000 7f908dffb640 1 -- 172.21.15.31:0/507791528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909802c030 con 0x555c6eb31b10 2024-09-16T00:47:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.190+0000 7f908dffb640 1 -- 172.21.15.31:0/507791528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9098003ad0 con 0x555c6eb31b10 2024-09-16T00:47:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.190+0000 7f908dffb640 1 -- 172.21.15.31:0/507791528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098012a70 con 0x555c6eb31b10 2024-09-16T00:47:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 -- 172.21.15.31:0/507791528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 msgr2=0x555c6eb3c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 --2- 172.21.15.31:0/507791528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb3c730 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f90980041a0 tx=0x7f909801e040 comp rx=0 tx=0).stop 2024-09-16T00:47:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 -- 172.21.15.31:0/507791528 shutdown_connections 2024-09-16T00:47:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 --2- 172.21.15.31:0/507791528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb3cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 --2- 172.21.15.31:0/507791528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb31b10 0x555c6eb3c730 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 --2- 172.21.15.31:0/507791528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb305a0 0x555c6eb47410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 -- 172.21.15.31:0/507791528 >> 172.21.15.31:0/507791528 conn(0x555c6eaf9310 msgr2=0x555c6eb36780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 -- 172.21.15.31:0/507791528 shutdown_connections 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 -- 172.21.15.31:0/507791528 wait complete. 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 -- start start 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.191+0000 7f90b0940740 1 Processor -- start 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90b0940740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb305a0 0x555c6eb43c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90b0940740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb31b10 0x555c6eb524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90b0940740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90b0940740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f909400ea30 con 0x555c6eb305a0 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90b0940740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f909400fea0 con 0x555c6eb3b340 2024-09-16T00:47:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90b0940740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f909400f100 con 0x555c6eb31b10 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb305a0 0x555c6eb43c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90b0940740 10 client.0 _ll_register_callbacks cb 0x7f909c00f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb305a0 0x555c6eb43c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46450/0 (socket says 172.21.15.31:46450) 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a77fe640 1 -- 172.21.15.31:0/1216109297 learned_addr learned my addr 172.21.15.31:0/1216109297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a7fff640 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb31b10 0x555c6eb524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a77fe640 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a7fff640 1 -- 172.21.15.31:0/1216109297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 msgr2=0x555c6eb529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a7fff640 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a7fff640 1 -- 172.21.15.31:0/1216109297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb305a0 msgr2=0x555c6eb43c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a7fff640 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb305a0 0x555c6eb43c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a7fff640 1 -- 172.21.15.31:0/1216109297 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c6eb531e0 con 0x555c6eb31b10 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a77fe640 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb305a0 0x555c6eb43c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90a7fff640 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb31b10 0x555c6eb524a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x555c6eb365e0 tx=0x7f9098010d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c355dc4768be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.192+0000 7f90877fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90877fe640 1 -- 172.21.15.31:0/1216109297 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909802c040 con 0x555c6eb31b10 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90877fe640 1 -- 172.21.15.31:0/1216109297 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f9098009d20 con 0x555c6eb31b10 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90b0940740 1 -- 172.21.15.31:0/1216109297 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9098003ad0 con 0x555c6eb31b10 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90877fe640 1 -- 172.21.15.31:0/1216109297 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f909803e050 con 0x555c6eb31b10 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90b0940740 10 client.25795 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90b0940740 1 -- 172.21.15.31:0/1216109297 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555c6ebb5bc0 con 0x555c6eb31b10 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90877fe640 1 -- 172.21.15.31:0/1216109297 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9098012810 con 0x555c6eb31b10 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90877fe640 1 -- 172.21.15.31:0/1216109297 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f909800f040 con 0x555c6eb31b10 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90b0940740 10 client.25795 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90b0940740 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90b0940740 20 client.25795 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.193+0000 7f90b0940740 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555c6eb4aed0 0x555c6ebb8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.194+0000 7f90b0940740 4 client.25795 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:47:46.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.194+0000 7f90b0940740 1 -- 172.21.15.31:0/1216109297 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9098003ad0 con 0x555c6eb4aed0 2024-09-16T00:47:46.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.194+0000 7f90a77fe640 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555c6eb4aed0 0x555c6ebb8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:46.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.194+0000 7f90a77fe640 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555c6eb4aed0 0x555c6ebb8040 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ed4390e02b30a8 server_cookie=fa930c9b2df90a9d in_seq=0 out_seq=0 2024-09-16T00:47:46.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.194+0000 7f90877fe640 10 client.25795 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:46.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90877fe640 1 -- 172.21.15.31:0/1216109297 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9098003ad0 con 0x555c6eb4aed0 2024-09-16T00:47:46.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90877fe640 10 client.25795 handle_command_reply: tid=0 2024-09-16T00:47:46.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 client.25795 shutdown 2024-09-16T00:47:46.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 -- 172.21.15.31:0/1216109297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb31b10 msgr2=0x555c6eb524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:46.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb31b10 0x555c6eb524a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x555c6eb365e0 tx=0x7f9098010d70 comp rx=0 tx=0).stop 2024-09-16T00:47:46.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 -- 172.21.15.31:0/1216109297 shutdown_connections 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c6eb3b340 0x555c6eb529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555c6eb4aed0 0x555c6ebb8040 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c6eb31b10 0x555c6eb524a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 --2- 172.21.15.31:0/1216109297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555c6eb305a0 0x555c6eb43c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 -- 172.21.15.31:0/1216109297 >> 172.21.15.31:0/1216109297 conn(0x555c6eaf9310 msgr2=0x555c6eb320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90877fe640 0 client.25795 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 -- 172.21.15.31:0/1216109297 shutdown_connections 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 1 -- 172.21.15.31:0/1216109297 wait complete. 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.196+0000 7f90b0940740 20 client.25795 trim_cache size 0 max 16384 2024-09-16T00:47:46.202 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:47:46.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90900a3390 msgr2=0x7f90900a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:46.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90900a3390 0x7f90900a5850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f90a009b380 tx=0x7f90a0097040 comp rx=0 tx=0).stop 2024-09-16T00:47:46.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90a81082a0 msgr2=0x7f90a819acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:46.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90a81082a0 0x7f90a819acf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f90a000b990 tx=0x7f90a000be50 comp rx=0 tx=0).stop 2024-09-16T00:47:46.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 shutdown_connections 2024-09-16T00:47:46.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90a81082a0 0x7f90a819acf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a81036e0 0x7f90a8196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f90900a3390 0x7f90900a5850 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 --2- 172.21.15.31:0/2946355490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a8102d30 0x7f90a8196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:46.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 >> 172.21.15.31:0/2946355490 conn(0x7f90a80fc7d0 msgr2=0x7f90a8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:46.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 shutdown_connections 2024-09-16T00:47:46.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:46.197+0000 7f90ae567640 1 -- 172.21.15.31:0/2946355490 wait complete. 2024-09-16T00:47:47.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:47 smithi031 ceph-mon[30956]: pgmap v555: 129 pgs: 129 active+clean; 600 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 767 op/s 2024-09-16T00:47:47.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:47 smithi081 ceph-mon[35539]: pgmap v555: 129 pgs: 129 active+clean; 600 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 767 op/s 2024-09-16T00:47:47.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:47 smithi102 ceph-mon[35364]: pgmap v555: 129 pgs: 129 active+clean; 600 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 767 op/s 2024-09-16T00:47:48.212 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:47:48.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:47:48.476 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:49.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:49 smithi031 ceph-mon[30956]: pgmap v556: 129 pgs: 129 active+clean; 597 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 827 op/s 2024-09-16T00:47:49.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:49 smithi081 ceph-mon[35539]: pgmap v556: 129 pgs: 129 active+clean; 597 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 827 op/s 2024-09-16T00:47:49.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:49 smithi102 ceph-mon[35364]: pgmap v556: 129 pgs: 129 active+clean; 597 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 827 op/s 2024-09-16T00:47:49.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.254+0000 7fd6b2203640 1 -- 172.21.15.31:0/4151339834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac102d90 msgr2=0x7fd6ac103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.254+0000 7fd6b2203640 1 --2- 172.21.15.31:0/4151339834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac102d90 0x7fd6ac103170 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009930 tx=0x7fd69c02a420 comp rx=0 tx=0).stop 2024-09-16T00:47:49.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.254+0000 7fd6b2203640 1 -- 172.21.15.31:0/4151339834 shutdown_connections 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.254+0000 7fd6b2203640 1 --2- 172.21.15.31:0/4151339834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ac108290 0x7fd6ac10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.254+0000 7fd6b2203640 1 --2- 172.21.15.31:0/4151339834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6ac103740 0x7fd6ac107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.254+0000 7fd6b2203640 1 --2- 172.21.15.31:0/4151339834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac102d90 0x7fd6ac103170 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.254+0000 7fd6b2203640 1 -- 172.21.15.31:0/4151339834 >> 172.21.15.31:0/4151339834 conn(0x7fd6ac0fc7d0 msgr2=0x7fd6ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.254+0000 7fd6b2203640 1 -- 172.21.15.31:0/4151339834 shutdown_connections 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.254+0000 7fd6b2203640 1 -- 172.21.15.31:0/4151339834 wait complete. 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.255+0000 7fd6b2203640 1 Processor -- start 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.255+0000 7fd6b2203640 1 -- start start 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.255+0000 7fd6b2203640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6ac102d90 0x7fd6ac1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.255+0000 7fd6b2203640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ac103740 0x7fd6ac198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.255+0000 7fd6b2203640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac108290 0x7fd6ac19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.255+0000 7fd6b2203640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac112090 con 0x7fd6ac108290 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.255+0000 7fd6b2203640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac111f10 con 0x7fd6ac103740 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.255+0000 7fd6b2203640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac112210 con 0x7fd6ac102d90 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6aaffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ac103740 0x7fd6ac198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac108290 0x7fd6ac19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6aaffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ac103740 0x7fd6ac198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:58126/0 (socket says 172.21.15.31:58126) 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6aaffd640 1 -- 172.21.15.31:0/3695734757 learned_addr learned my addr 172.21.15.31:0/3695734757 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6ac102d90 0x7fd6ac1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6aaffd640 1 -- 172.21.15.31:0/3695734757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6ac102d90 msgr2=0x7fd6ac1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6aaffd640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6ac102d90 0x7fd6ac1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6aaffd640 1 -- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac108290 msgr2=0x7fd6ac19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6aaffd640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac108290 0x7fd6ac19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6aaffd640 1 -- 172.21.15.31:0/3695734757 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac19d570 con 0x7fd6ac103740 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6abfff640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac108290 0x7fd6ac19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6aaffd640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ac103740 0x7fd6ac198b00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000d740 tx=0x7fd6a000dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6a8ff9640 1 -- 172.21.15.31:0/3695734757 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0017070 con 0x7fd6ac103740 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6b2203640 1 -- 172.21.15.31:0/3695734757 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac19d860 con 0x7fd6ac103740 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6a8ff9640 1 -- 172.21.15.31:0/3695734757 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a000de50 con 0x7fd6ac103740 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6a8ff9640 1 -- 172.21.15.31:0/3695734757 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004e00 con 0x7fd6ac103740 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.256+0000 7fd6b2203640 1 -- 172.21.15.31:0/3695734757 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ac1a5140 con 0x7fd6ac103740 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.259+0000 7fd6a8ff9640 1 -- 172.21.15.31:0/3695734757 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd6a0004fa0 con 0x7fd6ac103740 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.259+0000 7fd6b45dc740 1 Processor -- start 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6b45dc740 1 -- start start 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 0x561217422e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217431c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x561217434540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561217388280 con 0x561217423360 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56121732fb10 con 0x561217422a40 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561217384240 con 0x561217432150 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217431c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217431c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54084/0 (socket says 172.21.15.31:54084) 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6ab7fe640 1 -- 172.21.15.31:0/1051286848 learned_addr learned my addr 172.21.15.31:0/1051286848 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6abfff640 1 --2- 172.21.15.31:0/1051286848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 0x561217422e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6aaffd640 1 --2- 172.21.15.31:0/1051286848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x561217434540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6ab7fe640 1 -- 172.21.15.31:0/1051286848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 msgr2=0x561217434540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/1051286848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x561217434540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6ab7fe640 1 -- 172.21.15.31:0/1051286848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 msgr2=0x561217422e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/1051286848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 0x561217422e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6ab7fe640 1 -- 172.21.15.31:0/1051286848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121733c1f0 con 0x561217423360 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6a8ff9640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6940a32c0 0x7fd6940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.260+0000 7fd6aaffd640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6940a32c0 0x7fd6940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.261+0000 7fd6a8ff9640 1 -- 172.21.15.31:0/3695734757 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd6a00cf570 con 0x7fd6ac103740 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.261+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/1051286848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217431c10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd69c02a940 tx=0x7fd69c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fb0b8d997d7af3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.261+0000 7fd691ffb640 1 -- 172.21.15.31:0/1051286848 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c040070 con 0x561217423360 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.261+0000 7fd691ffb640 1 -- 172.21.15.31:0/1051286848 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd69c0318f0 con 0x561217423360 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.261+0000 7fd691ffb640 1 -- 172.21.15.31:0/1051286848 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c035040 con 0x561217423360 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.261+0000 7fd6b45dc740 1 -- 172.21.15.31:0/1051286848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 msgr2=0x561217431c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.261+0000 7fd6aaffd640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6940a32c0 0x7fd6940a5780 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00cd030 tx=0x7fd6a0086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.261+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/1051286848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217431c10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd69c02a940 tx=0x7fd69c005c00 comp rx=0 tx=0).stop 2024-09-16T00:47:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 -- 172.21.15.31:0/1051286848 shutdown_connections 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/1051286848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x561217434540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/1051286848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217431c10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/1051286848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 0x561217422e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 -- 172.21.15.31:0/1051286848 >> 172.21.15.31:0/1051286848 conn(0x56121741f0f0 msgr2=0x56121741f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 -- 172.21.15.31:0/1051286848 shutdown_connections 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 -- 172.21.15.31:0/1051286848 wait complete. 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 -- start start 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 Processor -- start 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x56121742f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 0x56121742f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x56121742faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561217434d20 con 0x561217422a40 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561217342f10 con 0x561217423360 2024-09-16T00:47:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.262+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561217434ea0 con 0x561217432150 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6b45dc740 10 client.0 _ll_register_callbacks cb 0x56121730ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x56121742f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x56121742f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54100/0 (socket says 172.21.15.31:54100) 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 -- 172.21.15.31:0/2557185700 learned_addr learned my addr 172.21.15.31:0/2557185700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 0x56121742f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x56121742faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 -- 172.21.15.31:0/2557185700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 msgr2=0x56121742faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x56121742faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 -- 172.21.15.31:0/2557185700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 msgr2=0x56121742f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 0x56121742f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 -- 172.21.15.31:0/2557185700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612170f96a0 con 0x561217422a40 2024-09-16T00:47:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6abfff640 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x56121742f020 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40127f0 tx=0x7fd6a4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e205753c237c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd68affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:47:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd68affd640 1 -- 172.21.15.31:0/2557185700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4004b20 con 0x561217422a40 2024-09-16T00:47:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd68affd640 1 -- 172.21.15.31:0/2557185700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd6a4004cc0 con 0x561217422a40 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2557185700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd69c0318f0 con 0x561217422a40 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd68affd640 1 -- 172.21.15.31:0/2557185700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd6a4025050 con 0x561217422a40 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6b45dc740 10 client.16203 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.263+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2557185700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5612174409b0 con 0x561217422a40 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.264+0000 7fd68affd640 1 -- 172.21.15.31:0/2557185700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a401c040 con 0x561217422a40 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.264+0000 7fd68affd640 1 -- 172.21.15.31:0/2557185700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd6a4004030 con 0x561217422a40 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.264+0000 7fd6b45dc740 10 client.16203 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.264+0000 7fd6b45dc740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.264+0000 7fd6b45dc740 20 client.16203 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.264+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561217441e70 0x5612174442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.264+0000 7fd6b45dc740 4 client.16203 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.264+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2557185700 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd69c0318f0 con 0x561217441e70 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.264+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561217441e70 0x5612174442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.265+0000 7fd68affd640 10 client.16203 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.265+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561217441e70 0x5612174442b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6809ec0826a80546 server_cookie=9103f2fb707d4f7e in_seq=0 out_seq=0 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd68affd640 1 -- 172.21.15.31:0/2557185700 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd69c0318f0 con 0x561217441e70 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd68affd640 10 client.16203 handle_command_reply: tid=0 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 client.16203 shutdown 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2557185700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 msgr2=0x56121742f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x56121742f020 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40127f0 tx=0x7fd6a4005bd0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2557185700 shutdown_connections 2024-09-16T00:47:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561217441e70 0x5612174442b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x56121742faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 0x56121742f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd68affd640 0 client.16203 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2557185700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x56121742f020 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2557185700 >> 172.21.15.31:0/2557185700 conn(0x56121741f0f0 msgr2=0x561217424e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2557185700 shutdown_connections 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2557185700 wait complete. 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.267+0000 7fd6b45dc740 20 client.16203 trim_cache size 0 max 16384 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6b45dc740 1 Processor -- start 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6b45dc740 1 -- start start 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x561217449bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 0x56121744a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x56121742c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac1aaa40 con 0x561217422a40 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac1ad2c0 con 0x561217423360 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac1ab0b0 con 0x561217432150 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x561217449bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x561217449bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54102/0 (socket says 172.21.15.31:54102) 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6abfff640 1 -- 172.21.15.31:0/3064355543 learned_addr learned my addr 172.21.15.31:0/3064355543 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 0x56121744a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6abfff640 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x56121742c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6abfff640 1 -- 172.21.15.31:0/3064355543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 msgr2=0x56121742c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6abfff640 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x56121742c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6abfff640 1 -- 172.21.15.31:0/3064355543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 msgr2=0x56121744a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6abfff640 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 0x56121744a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6abfff640 1 -- 172.21.15.31:0/3064355543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121741e980 con 0x561217422a40 2024-09-16T00:47:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.285+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 0x56121744a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.286+0000 7fd6abfff640 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x561217449bd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009730 tx=0x7fd6a400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e9ec9315b050d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.286+0000 7fd6917fa640 1 -- 172.21.15.31:0/3064355543 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4012290 con 0x561217422a40 2024-09-16T00:47:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.286+0000 7fd6917fa640 1 -- 172.21.15.31:0/3064355543 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a40046c0 con 0x561217422a40 2024-09-16T00:47:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.286+0000 7fd6917fa640 1 -- 172.21.15.31:0/3064355543 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4003bb0 con 0x561217422a40 2024-09-16T00:47:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.286+0000 7fd6b45dc740 1 -- 172.21.15.31:0/3064355543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 msgr2=0x561217449bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.286+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x561217449bd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009730 tx=0x7fd6a400f3c0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 -- 172.21.15.31:0/3064355543 shutdown_connections 2024-09-16T00:47:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x56121742c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217423360 0x56121744a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/3064355543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217422a40 0x561217449bd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 -- 172.21.15.31:0/3064355543 >> 172.21.15.31:0/3064355543 conn(0x56121741f0f0 msgr2=0x56121741e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 -- 172.21.15.31:0/3064355543 shutdown_connections 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 -- 172.21.15.31:0/3064355543 wait complete. 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 -- start start 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 Processor -- start 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 0x56121743a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217436910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x561217436e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac1ab780 con 0x561217423360 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac1acbf0 con 0x561217422a40 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac1abe50 con 0x561217432150 2024-09-16T00:47:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.287+0000 7fd6b45dc740 10 client.0 _ll_register_callbacks cb 0x7fd6a400f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217436910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 0x56121743a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217436910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54112/0 (socket says 172.21.15.31:54112) 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6ab7fe640 1 -- 172.21.15.31:0/2205988088 learned_addr learned my addr 172.21.15.31:0/2205988088 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6abfff640 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x561217436e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6ab7fe640 1 -- 172.21.15.31:0/2205988088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 msgr2=0x561217436e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x561217436e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6ab7fe640 1 -- 172.21.15.31:0/2205988088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 msgr2=0x56121743a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 0x56121743a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6ab7fe640 1 -- 172.21.15.31:0/2205988088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561217437390 con 0x561217423360 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6ab7fe640 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217436910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005540 tx=0x7fd69c0054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ca21f10fcde5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd68affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd68affd640 1 -- 172.21.15.31:0/2205988088 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c040040 con 0x561217423360 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2205988088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a40046c0 con 0x561217423360 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd68affd640 1 -- 172.21.15.31:0/2205988088 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fd69c004240 con 0x561217423360 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd68affd640 1 -- 172.21.15.31:0/2205988088 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd69c05d050 con 0x561217423360 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6b45dc740 10 client.16209 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.288+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2205988088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5612174ad8d0 con 0x561217423360 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd68affd640 1 -- 172.21.15.31:0/2205988088 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a40046c0 con 0x561217423360 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd68affd640 1 -- 172.21.15.31:0/2205988088 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd69c04f3b0 con 0x561217423360 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd6b45dc740 10 client.16209 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd6b45dc740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd6b45dc740 20 client.16209 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5612174add20 0x5612174b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd6b45dc740 4 client.16209 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2205988088 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5612173957f0 con 0x5612174add20 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd6abfff640 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5612174add20 0x5612174b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd6abfff640 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5612174add20 0x5612174b0110 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5563a114aa728e1 server_cookie=55785bdccbce0327 in_seq=0 out_seq=0 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.289+0000 7fd68affd640 10 client.16209 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd68affd640 1 -- 172.21.15.31:0/2205988088 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5612173957f0 con 0x5612174add20 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd68affd640 10 client.16209 handle_command_reply: tid=0 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 client.16209 shutdown 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2205988088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 msgr2=0x561217436910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217436910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005540 tx=0x7fd69c0054d0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2205988088 shutdown_connections 2024-09-16T00:47:49.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x5612174add20 0x5612174b0110 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.577 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:47:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:47:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "6cfd8eb7-2b9f-4855-8387-fd510542a79d", 2024-09-16T00:47:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:47:49.577 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:47:49.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561217432150 0x561217436e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561217423360 0x561217436910 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd68affd640 0 client.16209 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 --2- 172.21.15.31:0/2205988088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561217422a40 0x56121743a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2205988088 >> 172.21.15.31:0/2205988088 conn(0x56121741f0f0 msgr2=0x5612174245a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2205988088 shutdown_connections 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.333+0000 7fd6b45dc740 1 -- 172.21.15.31:0/2205988088 wait complete. 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b45dc740 20 client.16209 trim_cache size 0 max 16384 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 -- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6940a32c0 msgr2=0x7fd6940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6940a32c0 0x7fd6940a5780 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00cd030 tx=0x7fd6a0086040 comp rx=0 tx=0).stop 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 -- 172.21.15.31:0/3695734757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ac103740 msgr2=0x7fd6ac198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ac103740 0x7fd6ac198b00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000d740 tx=0x7fd6a000dc00 comp rx=0 tx=0).stop 2024-09-16T00:47:49.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 -- 172.21.15.31:0/3695734757 shutdown_connections 2024-09-16T00:47:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac108290 0x7fd6ac19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ac103740 0x7fd6ac198b00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fd6940a32c0 0x7fd6940a5780 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 --2- 172.21.15.31:0/3695734757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6ac102d90 0x7fd6ac1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 -- 172.21.15.31:0/3695734757 >> 172.21.15.31:0/3695734757 conn(0x7fd6ac0fc7d0 msgr2=0x7fd6ac105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 -- 172.21.15.31:0/3695734757 shutdown_connections 2024-09-16T00:47:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:49.334+0000 7fd6b2203640 1 -- 172.21.15.31:0/3695734757 wait complete. 2024-09-16T00:47:50.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:47:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:47:51.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:51 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:47:51.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:51 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:47:51.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:51 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:47:51.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:51 smithi031 ceph-mon[30956]: pgmap v557: 129 pgs: 129 active+clean; 585 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 783 op/s 2024-09-16T00:47:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.469+0000 7f275265d640 1 -- 172.21.15.31:0/748402184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 msgr2=0x7f274c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.469+0000 7f275265d640 1 --2- 172.21.15.31:0/748402184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 0x7f274c1021c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f273c009900 tx=0x7f273c02a410 comp rx=0 tx=0).stop 2024-09-16T00:47:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.469+0000 7f275265d640 1 -- 172.21.15.31:0/748402184 shutdown_connections 2024-09-16T00:47:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.469+0000 7f275265d640 1 --2- 172.21.15.31:0/748402184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274c106f10 0x7f274c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.469+0000 7f275265d640 1 --2- 172.21.15.31:0/748402184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274c102790 0x7f274c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.469+0000 7f275265d640 1 --2- 172.21.15.31:0/748402184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 0x7f274c1021c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.469+0000 7f275265d640 1 -- 172.21.15.31:0/748402184 >> 172.21.15.31:0/748402184 conn(0x7f274c0fd880 msgr2=0x7f274c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.469+0000 7f275265d640 1 -- 172.21.15.31:0/748402184 shutdown_connections 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.469+0000 7f275265d640 1 -- 172.21.15.31:0/748402184 wait complete. 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f275265d640 1 Processor -- start 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f275265d640 1 -- start start 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f275265d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 0x7f274c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f275265d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274c102790 0x7f274c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f275265d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274c106f10 0x7f274c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f275265d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f274c111f70 con 0x7f274c101de0 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f275265d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f274c111df0 con 0x7f274c106f10 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f275265d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f274c1120f0 con 0x7f274c102790 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f274bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 0x7f274c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f274bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 0x7f274c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54128/0 (socket says 172.21.15.31:54128) 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.470+0000 7f274bfff640 1 -- 172.21.15.31:0/1325386576 learned_addr learned my addr 172.21.15.31:0/1325386576 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f274b7fe640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274c102790 0x7f274c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f2750bfa640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274c106f10 0x7f274c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f274bfff640 1 -- 172.21.15.31:0/1325386576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274c102790 msgr2=0x7f274c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f274bfff640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274c102790 0x7f274c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f274bfff640 1 -- 172.21.15.31:0/1325386576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274c106f10 msgr2=0x7f274c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f274bfff640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274c106f10 0x7f274c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f274bfff640 1 -- 172.21.15.31:0/1325386576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c19f660 con 0x7f274c101de0 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f274b7fe640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274c102790 0x7f274c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f274bfff640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 0x7f274c19a6b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f273c006010 tx=0x7f273c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f27497fa640 1 -- 172.21.15.31:0/1325386576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c03f030 con 0x7f274c101de0 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f275265d640 1 -- 172.21.15.31:0/1325386576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c19f8f0 con 0x7f274c101de0 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f27497fa640 1 -- 172.21.15.31:0/1325386576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f273c004440 con 0x7f274c101de0 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f275265d640 1 -- 172.21.15.31:0/1325386576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c1a7230 con 0x7f274c101de0 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.471+0000 7f27497fa640 1 -- 172.21.15.31:0/1325386576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c035070 con 0x7f274c101de0 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.474+0000 7f27497fa640 1 -- 172.21.15.31:0/1325386576 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f273c042030 con 0x7f274c101de0 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.474+0000 7f2754a36740 1 Processor -- start 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.474+0000 7f2754a36740 1 -- start start 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.474+0000 7f2754a36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aaf4a0 0x555858aaf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.474+0000 7f2754a36740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aafdc0 0x555858aba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.474+0000 7f2754a36740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858abca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.474+0000 7f2754a36740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555858a152e0 con 0x555858aafdc0 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.474+0000 7f2754a36740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5558589bcce0 con 0x555858aba680 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.474+0000 7f2754a36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555858a112a0 con 0x555858aaf4a0 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f27497fa640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f27340a3240 0x7f27340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858abca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aafdc0 0x555858aba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858abca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:58170/0 (socket says 172.21.15.31:58170) 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274b7fe640 1 -- 172.21.15.31:0/939385820 learned_addr learned my addr 172.21.15.31:0/939385820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f2750bfa640 1 --2- 172.21.15.31:0/939385820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aaf4a0 0x555858aaf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f2750bfa640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f27340a3240 0x7f27340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f27497fa640 1 -- 172.21.15.31:0/1325386576 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f273c0efc80 con 0x7f274c101de0 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274b7fe640 1 -- 172.21.15.31:0/939385820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aaf4a0 msgr2=0x555858aaf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274b7fe640 1 --2- 172.21.15.31:0/939385820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aaf4a0 0x555858aaf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274b7fe640 1 -- 172.21.15.31:0/939385820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aafdc0 msgr2=0x555858aba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274b7fe640 1 --2- 172.21.15.31:0/939385820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aafdc0 0x555858aba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274b7fe640 1 -- 172.21.15.31:0/939385820 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558589a1440 con 0x555858aba680 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f2750bfa640 1 --2- 172.21.15.31:0/939385820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aaf4a0 0x555858aaf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f274b7fe640 1 --2- 172.21.15.31:0/939385820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858abca70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f27400114f0 tx=0x7f27400119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc14b35375c909a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f2750bfa640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f27340a3240 0x7f27340a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2744005fd0 tx=0x7f2744009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f27327fc640 1 -- 172.21.15.31:0/939385820 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740004020 con 0x555858aba680 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f27327fc640 1 -- 172.21.15.31:0/939385820 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27400041c0 con 0x555858aba680 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.475+0000 7f27327fc640 1 -- 172.21.15.31:0/939385820 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274000f960 con 0x555858aba680 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 -- 172.21.15.31:0/939385820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 msgr2=0x555858abca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 --2- 172.21.15.31:0/939385820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858abca70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f27400114f0 tx=0x7f27400119b0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 -- 172.21.15.31:0/939385820 shutdown_connections 2024-09-16T00:47:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 --2- 172.21.15.31:0/939385820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858abca70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 --2- 172.21.15.31:0/939385820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aafdc0 0x555858aba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 --2- 172.21.15.31:0/939385820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aaf4a0 0x555858aaf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 -- 172.21.15.31:0/939385820 >> 172.21.15.31:0/939385820 conn(0x555858aaba30 msgr2=0x555858aabe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 -- 172.21.15.31:0/939385820 shutdown_connections 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 -- 172.21.15.31:0/939385820 wait complete. 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.476+0000 7f2754a36740 1 -- start start 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f2754a36740 1 Processor -- start 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f2754a36740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f2754a36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ab71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f2754a36740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ab76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f2754a36740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555858abd250 con 0x555858aaf4a0 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f2754a36740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5558589d0050 con 0x555858aba680 2024-09-16T00:47:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f2754a36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555858abd3d0 con 0x555858aafdc0 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f2754a36740 10 client.0 _ll_register_callbacks cb 0x555858a10160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f274bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f274bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54152/0 (socket says 172.21.15.31:54152) 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f274bfff640 1 -- 172.21.15.31:0/877287622 learned_addr learned my addr 172.21.15.31:0/877287622 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f274b7fe640 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ab71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f2750bfa640 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ab76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f274bfff640 1 -- 172.21.15.31:0/877287622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 msgr2=0x555858ab71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f274bfff640 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ab71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f274bfff640 1 -- 172.21.15.31:0/877287622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 msgr2=0x555858ab76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f274bfff640 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ab76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.477+0000 7f274bfff640 1 -- 172.21.15.31:0/877287622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558587863c0 con 0x555858aaf4a0 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f274bfff640 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab9ef0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f273c0ed740 tx=0x7f273c08bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a74cf0d4bf5dd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f272b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f272b7fe640 1 -- 172.21.15.31:0/877287622 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c049040 con 0x555858aaf4a0 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f272b7fe640 1 -- 172.21.15.31:0/877287622 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f273c0b8610 con 0x555858aaf4a0 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f2754a36740 1 -- 172.21.15.31:0/877287622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27400041c0 con 0x555858aaf4a0 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f272b7fe640 1 -- 172.21.15.31:0/877287622 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f273c0bbfa0 con 0x555858aaf4a0 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f2754a36740 10 client.16221 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f2754a36740 1 -- 172.21.15.31:0/877287622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555858ac8b40 con 0x555858aaf4a0 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f272b7fe640 1 -- 172.21.15.31:0/877287622 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27400041c0 con 0x555858aaf4a0 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f272b7fe640 1 -- 172.21.15.31:0/877287622 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f273c0a6040 con 0x555858aaf4a0 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f2754a36740 10 client.16221 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f2754a36740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f2754a36740 20 client.16221 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f2754a36740 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555858aca040 0x555858acc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f2754a36740 4 client.16221 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.478+0000 7f2754a36740 1 -- 172.21.15.31:0/877287622 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555858a22850 con 0x555858aca040 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.479+0000 7f274b7fe640 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555858aca040 0x555858acc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.479+0000 7f274b7fe640 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555858aca040 0x555858acc490 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec79552071ac293 server_cookie=1dbd9e6ba0e16f61 in_seq=0 out_seq=0 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.479+0000 7f272b7fe640 10 client.16221 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.480+0000 7f272b7fe640 1 -- 172.21.15.31:0/877287622 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555858a22850 con 0x555858aca040 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.480+0000 7f272b7fe640 10 client.16221 handle_command_reply: tid=0 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.480+0000 7f2754a36740 1 client.16221 shutdown 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 -- 172.21.15.31:0/877287622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 msgr2=0x555858ab9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab9ef0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f273c0ed740 tx=0x7f273c08bcd0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 -- 172.21.15.31:0/877287622 shutdown_connections 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ab76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555858aca040 0x555858acc490 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ab71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 --2- 172.21.15.31:0/877287622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab9ef0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 -- 172.21.15.31:0/877287622 >> 172.21.15.31:0/877287622 conn(0x555858aaba30 msgr2=0x555858ab1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f272b7fe640 0 client.16221 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 -- 172.21.15.31:0/877287622 shutdown_connections 2024-09-16T00:47:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 1 -- 172.21.15.31:0/877287622 wait complete. 2024-09-16T00:47:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.481+0000 7f2754a36740 20 client.16221 trim_cache size 0 max 16384 2024-09-16T00:47:51.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:51 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:47:51.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:51 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:47:51.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:51 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:47:51.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:51 smithi081 ceph-mon[35539]: pgmap v557: 129 pgs: 129 active+clean; 585 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 783 op/s 2024-09-16T00:47:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f2754a36740 1 Processor -- start 2024-09-16T00:47:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f2754a36740 1 -- start start 2024-09-16T00:47:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f2754a36740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f2754a36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ab2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f2754a36740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ad1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f2754a36740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f274c1aeec0 con 0x555858aaf4a0 2024-09-16T00:47:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f2754a36740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f274c1b1740 con 0x555858aba680 2024-09-16T00:47:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f2754a36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f274c1af530 con 0x555858aafdc0 2024-09-16T00:47:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f274bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f274b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ab2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f274bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54168/0 (socket says 172.21.15.31:54168) 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f274bfff640 1 -- 172.21.15.31:0/1504419699 learned_addr learned my addr 172.21.15.31:0/1504419699 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f2750bfa640 1 --2- 172.21.15.31:0/1504419699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ad1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f274b7fe640 1 -- 172.21.15.31:0/1504419699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 msgr2=0x555858ad1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f274b7fe640 1 --2- 172.21.15.31:0/1504419699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ad1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f274b7fe640 1 -- 172.21.15.31:0/1504419699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 msgr2=0x555858ab7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f274b7fe640 1 --2- 172.21.15.31:0/1504419699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.500+0000 7f274b7fe640 1 -- 172.21.15.31:0/1504419699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555858ab6150 con 0x555858aafdc0 2024-09-16T00:47:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.501+0000 7f274b7fe640 1 --2- 172.21.15.31:0/1504419699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ab2740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2740011eb0 tx=0x7f274000f5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=186a3f256d06cc09 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.501+0000 7f2731ffb640 1 -- 172.21.15.31:0/1504419699 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740004100 con 0x555858aafdc0 2024-09-16T00:47:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.501+0000 7f2731ffb640 1 -- 172.21.15.31:0/1504419699 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27400042a0 con 0x555858aafdc0 2024-09-16T00:47:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.501+0000 7f2731ffb640 1 -- 172.21.15.31:0/1504419699 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740003b80 con 0x555858aafdc0 2024-09-16T00:47:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 -- 172.21.15.31:0/1504419699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 msgr2=0x555858ab2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 --2- 172.21.15.31:0/1504419699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ab2740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2740011eb0 tx=0x7f274000f5c0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 -- 172.21.15.31:0/1504419699 shutdown_connections 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 --2- 172.21.15.31:0/1504419699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ad1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 --2- 172.21.15.31:0/1504419699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ab2740 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 --2- 172.21.15.31:0/1504419699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ab7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 -- 172.21.15.31:0/1504419699 >> 172.21.15.31:0/1504419699 conn(0x555858aaba30 msgr2=0x555858ab2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 -- 172.21.15.31:0/1504419699 shutdown_connections 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 -- 172.21.15.31:0/1504419699 wait complete. 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.502+0000 7f2754a36740 1 -- start start 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f2754a36740 1 Processor -- start 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f2754a36740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ac2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f2754a36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ac2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f2754a36740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ab3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f2754a36740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f274c1afc00 con 0x555858aaf4a0 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f2754a36740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f274c1b1070 con 0x555858aba680 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f2754a36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f274c1b02d0 con 0x555858aafdc0 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f2754a36740 10 client.0 _ll_register_callbacks cb 0x7f273c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f274bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ac2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f274bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ac2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54180/0 (socket says 172.21.15.31:54180) 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f274bfff640 1 -- 172.21.15.31:0/3915714566 learned_addr learned my addr 172.21.15.31:0/3915714566 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f2750bfa640 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ab3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f274b7fe640 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ac2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.503+0000 7f274bfff640 1 -- 172.21.15.31:0/3915714566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 msgr2=0x555858ac2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f274bfff640 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ac2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f274bfff640 1 -- 172.21.15.31:0/3915714566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 msgr2=0x555858ab3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:47:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f274bfff640 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ab3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f274bfff640 1 -- 172.21.15.31:0/3915714566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555858ab4200 con 0x555858aaf4a0 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f2750bfa640 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ab3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f274b7fe640 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ac2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f274bfff640 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ac2800 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f273c0bc470 tx=0x7f273c0dc0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602171d7e6edd9a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f272b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f272b7fe640 1 -- 172.21.15.31:0/3915714566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c049040 con 0x555858aaf4a0 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f272b7fe640 1 -- 172.21.15.31:0/3915714566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f273c031dd0 con 0x555858aaf4a0 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f2754a36740 1 -- 172.21.15.31:0/3915714566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27400042a0 con 0x555858aaf4a0 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f272b7fe640 1 -- 172.21.15.31:0/3915714566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f273c0bbae0 con 0x555858aaf4a0 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f2754a36740 10 client.16227 fetch_fsmap learned FSMap version 20 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f2754a36740 1 -- 172.21.15.31:0/3915714566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555858ab4200 con 0x555858aaf4a0 2024-09-16T00:47:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.504+0000 7f272b7fe640 1 -- 172.21.15.31:0/3915714566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27400042a0 con 0x555858aaf4a0 2024-09-16T00:47:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f272b7fe640 1 -- 172.21.15.31:0/3915714566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f273c0a6040 con 0x555858aaf4a0 2024-09-16T00:47:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f2754a36740 10 client.16227 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:47:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f2754a36740 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:47:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f2754a36740 20 client.16227 populate_metadata read hostname 'smithi031' 2024-09-16T00:47:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f2754a36740 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555858b34a40 0x555858b36e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f2754a36740 4 client.16227 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:47:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f2754a36740 1 -- 172.21.15.31:0/3915714566 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555858ab7a70 con 0x555858b34a40 2024-09-16T00:47:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f274b7fe640 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555858b34a40 0x555858b36e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f274b7fe640 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555858b34a40 0x555858b36e60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7341018394c4bc2 server_cookie=e2349404caf4d0f7 in_seq=0 out_seq=0 2024-09-16T00:47:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.505+0000 7f272b7fe640 10 client.16227 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f272b7fe640 1 -- 172.21.15.31:0/3915714566 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x555858ab7a70 con 0x555858b34a40 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f272b7fe640 10 client.16227 handle_command_reply: tid=0 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 client.16227 shutdown 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 -- 172.21.15.31:0/3915714566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 msgr2=0x555858ac2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ac2800 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f273c0bc470 tx=0x7f273c0dc0b0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 -- 172.21.15.31:0/3915714566 shutdown_connections 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555858aba680 0x555858ab3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x555858b34a40 0x555858b36e60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555858aafdc0 0x555858ac2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 --2- 172.21.15.31:0/3915714566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555858aaf4a0 0x555858ac2800 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 -- 172.21.15.31:0/3915714566 >> 172.21.15.31:0/3915714566 conn(0x555858aaba30 msgr2=0x555858aaab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f272b7fe640 0 client.16227 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 -- 172.21.15.31:0/3915714566 shutdown_connections 2024-09-16T00:47:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 1 -- 172.21.15.31:0/3915714566 wait complete. 2024-09-16T00:47:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.507+0000 7f2754a36740 20 client.16227 trim_cache size 0 max 16384 2024-09-16T00:47:51.512 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:47:51.512 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (16317 inodes in the stack)", 2024-09-16T00:47:51.512 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:47:51.512 INFO:teuthology.orchestra.run.smithi031.stdout: "6cfd8eb7-2b9f-4855-8387-fd510542a79d": { 2024-09-16T00:47:51.512 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:47:51.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "6cfd8eb7-2b9f-4855-8387-fd510542a79d", 2024-09-16T00:47:51.512 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:47:51.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:51.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:47:51.513 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:47:51.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 -- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f27340a3240 msgr2=0x7f27340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:51.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f27340a3240 0x7f27340a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2744005fd0 tx=0x7f2744009500 comp rx=0 tx=0).stop 2024-09-16T00:47:51.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 -- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 msgr2=0x7f274c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:51.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 0x7f274c19a6b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f273c006010 tx=0x7f273c004040 comp rx=0 tx=0).stop 2024-09-16T00:47:51.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 -- 172.21.15.31:0/1325386576 shutdown_connections 2024-09-16T00:47:51.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274c106f10 0x7f274c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f27340a3240 0x7f27340a5700 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274c102790 0x7f274c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 --2- 172.21.15.31:0/1325386576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274c101de0 0x7f274c19a6b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:51.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 -- 172.21.15.31:0/1325386576 >> 172.21.15.31:0/1325386576 conn(0x7f274c0fd880 msgr2=0x7f274c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:51.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 -- 172.21.15.31:0/1325386576 shutdown_connections 2024-09-16T00:47:51.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:47:51.508+0000 7f275265d640 1 -- 172.21.15.31:0/1325386576 wait complete. 2024-09-16T00:47:51.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:51 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:47:51.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:51 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:47:51.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:51 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:47:51.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:51 smithi102 ceph-mon[35364]: pgmap v557: 129 pgs: 129 active+clean; 585 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 783 op/s 2024-09-16T00:47:52.989 INFO:tasks.cephfs.filesystem:scrub status for tag:6cfd8eb7-2b9f-4855-8387-fd510542a79d - {'path': '/', 'tag': '6cfd8eb7-2b9f-4855-8387-fd510542a79d', 'options': 'recursive,force'} 2024-09-16T00:47:53.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:53 smithi031 ceph-mon[30956]: pgmap v558: 129 pgs: 129 active+clean; 574 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 815 op/s 2024-09-16T00:47:53.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:53 smithi081 ceph-mon[35539]: pgmap v558: 129 pgs: 129 active+clean; 574 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 815 op/s 2024-09-16T00:47:53.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:53 smithi102 ceph-mon[35364]: pgmap v558: 129 pgs: 129 active+clean; 574 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 815 op/s 2024-09-16T00:47:55.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:55 smithi031 ceph-mon[30956]: pgmap v559: 129 pgs: 129 active+clean; 539 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 1.06k op/s 2024-09-16T00:47:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:55 smithi081 ceph-mon[35539]: pgmap v559: 129 pgs: 129 active+clean; 539 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 1.06k op/s 2024-09-16T00:47:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:55 smithi102 ceph-mon[35364]: pgmap v559: 129 pgs: 129 active+clean; 539 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 1.06k op/s 2024-09-16T00:47:57.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:57 smithi031 ceph-mon[30956]: pgmap v560: 129 pgs: 129 active+clean; 533 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 902 op/s 2024-09-16T00:47:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:57 smithi081 ceph-mon[35539]: pgmap v560: 129 pgs: 129 active+clean; 533 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 902 op/s 2024-09-16T00:47:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:57 smithi102 ceph-mon[35364]: pgmap v560: 129 pgs: 129 active+clean; 533 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 902 op/s 2024-09-16T00:47:59.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:47:59 smithi031 ceph-mon[30956]: pgmap v561: 129 pgs: 129 active+clean; 524 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 1.14k op/s 2024-09-16T00:47:59.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:47:59 smithi081 ceph-mon[35539]: pgmap v561: 129 pgs: 129 active+clean; 524 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 1.14k op/s 2024-09-16T00:47:59.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:47:59 smithi102 ceph-mon[35364]: pgmap v561: 129 pgs: 129 active+clean; 524 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 1.14k op/s 2024-09-16T00:48:01.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: pgmap v562: 129 pgs: 129 active+clean; 515 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.31k op/s 2024-09-16T00:48:01.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:48:01.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.377 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:48:01.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:48:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: pgmap v562: 129 pgs: 129 active+clean; 515 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.31k op/s 2024-09-16T00:48:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:48:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:48:01.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:48:01.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: pgmap v562: 129 pgs: 129 active+clean; 515 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.31k op/s 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:48:01.551 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:48:01.551 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:48:01.551 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:48:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:02 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-16T00:48:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:02 smithi031 ceph-mon[30956]: Unable to set osd_memory_target on smithi102 to 184587468: error parsing value: Value '184587468' is below minimum 939524096 2024-09-16T00:48:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:02 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-16T00:48:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:02 smithi031 ceph-mon[30956]: Unable to set osd_memory_target on smithi031 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-16T00:48:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:02 smithi031 ceph-mon[30956]: Adjusting osd_memory_target on smithi081 to 176.0M 2024-09-16T00:48:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:02 smithi031 ceph-mon[30956]: Unable to set osd_memory_target on smithi081 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-16T00:48:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:02 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-16T00:48:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:02 smithi081 ceph-mon[35539]: Unable to set osd_memory_target on smithi102 to 184587468: error parsing value: Value '184587468' is below minimum 939524096 2024-09-16T00:48:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:02 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-16T00:48:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:02 smithi081 ceph-mon[35539]: Unable to set osd_memory_target on smithi031 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-16T00:48:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:02 smithi081 ceph-mon[35539]: Adjusting osd_memory_target on smithi081 to 176.0M 2024-09-16T00:48:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:02 smithi081 ceph-mon[35539]: Unable to set osd_memory_target on smithi081 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-16T00:48:02.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:02 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-16T00:48:02.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:02 smithi102 ceph-mon[35364]: Unable to set osd_memory_target on smithi102 to 184587468: error parsing value: Value '184587468' is below minimum 939524096 2024-09-16T00:48:02.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:02 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-16T00:48:02.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:02 smithi102 ceph-mon[35364]: Unable to set osd_memory_target on smithi031 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-16T00:48:02.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:02 smithi102 ceph-mon[35364]: Adjusting osd_memory_target on smithi081 to 176.0M 2024-09-16T00:48:02.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:02 smithi102 ceph-mon[35364]: Unable to set osd_memory_target on smithi081 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-16T00:48:03.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:03 smithi031 ceph-mon[30956]: pgmap v563: 129 pgs: 129 active+clean; 506 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 1.42k op/s 2024-09-16T00:48:03.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:03 smithi081 ceph-mon[35539]: pgmap v563: 129 pgs: 129 active+clean; 506 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 1.42k op/s 2024-09-16T00:48:03.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:03 smithi102 ceph-mon[35364]: pgmap v563: 129 pgs: 129 active+clean; 506 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 1.42k op/s 2024-09-16T00:48:05.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:05 smithi031 ceph-mon[30956]: pgmap v564: 129 pgs: 129 active+clean; 485 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 1.70k op/s 2024-09-16T00:48:05.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:05 smithi081 ceph-mon[35539]: pgmap v564: 129 pgs: 129 active+clean; 485 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 1.70k op/s 2024-09-16T00:48:05.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:05 smithi102 ceph-mon[35364]: pgmap v564: 129 pgs: 129 active+clean; 485 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 1.70k op/s 2024-09-16T00:48:07.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:07 smithi031 ceph-mon[30956]: pgmap v565: 129 pgs: 129 active+clean; 477 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 1.45k op/s 2024-09-16T00:48:07.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:07 smithi081 ceph-mon[35539]: pgmap v565: 129 pgs: 129 active+clean; 477 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 1.45k op/s 2024-09-16T00:48:07.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:07 smithi102 ceph-mon[35364]: pgmap v565: 129 pgs: 129 active+clean; 477 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 1.45k op/s 2024-09-16T00:48:09.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:09 smithi031 ceph-mon[30956]: pgmap v566: 129 pgs: 129 active+clean; 455 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 1.55k op/s 2024-09-16T00:48:09.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:09 smithi081 ceph-mon[35539]: pgmap v566: 129 pgs: 129 active+clean; 455 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 1.55k op/s 2024-09-16T00:48:09.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:09 smithi102 ceph-mon[35364]: pgmap v566: 129 pgs: 129 active+clean; 455 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 1.55k op/s 2024-09-16T00:48:10.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:10 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:48:10.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:10 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:48:10.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:10 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:48:11.451 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:11 smithi102 ceph-mon[35364]: pgmap v567: 129 pgs: 129 active+clean; 439 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.48k op/s 2024-09-16T00:48:11.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:11 smithi081 ceph-mon[35539]: pgmap v567: 129 pgs: 129 active+clean; 439 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.48k op/s 2024-09-16T00:48:11.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:11 smithi031 ceph-mon[30956]: pgmap v567: 129 pgs: 129 active+clean; 439 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.48k op/s 2024-09-16T00:48:13.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:13 smithi081 ceph-mon[35539]: pgmap v568: 129 pgs: 129 active+clean; 432 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 1.36k op/s 2024-09-16T00:48:13.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:13 smithi102 ceph-mon[35364]: pgmap v568: 129 pgs: 129 active+clean; 432 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 1.36k op/s 2024-09-16T00:48:13.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:13 smithi031 ceph-mon[30956]: pgmap v568: 129 pgs: 129 active+clean; 432 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 1.36k op/s 2024-09-16T00:48:15.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:15 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:48:15.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:15 smithi081 ceph-mon[35539]: pgmap v569: 129 pgs: 129 active+clean; 414 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 1.39k op/s 2024-09-16T00:48:15.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:15 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:48:15.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:15 smithi102 ceph-mon[35364]: pgmap v569: 129 pgs: 129 active+clean; 414 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 1.39k op/s 2024-09-16T00:48:15.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:15 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:48:15.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:15 smithi031 ceph-mon[30956]: pgmap v569: 129 pgs: 129 active+clean; 414 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 1.39k op/s 2024-09-16T00:48:17.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:17 smithi081 ceph-mon[35539]: pgmap v570: 129 pgs: 129 active+clean; 407 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 1.04k op/s 2024-09-16T00:48:17.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:17 smithi102 ceph-mon[35364]: pgmap v570: 129 pgs: 129 active+clean; 407 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 1.04k op/s 2024-09-16T00:48:17.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:17 smithi031 ceph-mon[30956]: pgmap v570: 129 pgs: 129 active+clean; 407 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 1.04k op/s 2024-09-16T00:48:19.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:19 smithi081 ceph-mon[35539]: pgmap v571: 129 pgs: 129 active+clean; 387 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 1.01k op/s 2024-09-16T00:48:19.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:19 smithi102 ceph-mon[35364]: pgmap v571: 129 pgs: 129 active+clean; 387 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 1.01k op/s 2024-09-16T00:48:19.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:19 smithi031 ceph-mon[30956]: pgmap v571: 129 pgs: 129 active+clean; 387 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 1.01k op/s 2024-09-16T00:48:21.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:21 smithi081 ceph-mon[35539]: pgmap v572: 129 pgs: 129 active+clean; 366 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 845 op/s 2024-09-16T00:48:21.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:21 smithi102 ceph-mon[35364]: pgmap v572: 129 pgs: 129 active+clean; 366 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 845 op/s 2024-09-16T00:48:21.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:21 smithi031 ceph-mon[30956]: pgmap v572: 129 pgs: 129 active+clean; 366 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 845 op/s 2024-09-16T00:48:22.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:48:23.369 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:48:23.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:23 smithi031 ceph-mon[30956]: pgmap v573: 129 pgs: 129 active+clean; 361 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 704 op/s 2024-09-16T00:48:23.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:23 smithi081 ceph-mon[35539]: pgmap v573: 129 pgs: 129 active+clean; 361 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 704 op/s 2024-09-16T00:48:23.552 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:23 smithi102 ceph-mon[35364]: pgmap v573: 129 pgs: 129 active+clean; 361 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 704 op/s 2024-09-16T00:48:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 -- 172.21.15.31:0/3222206817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170106a60 msgr2=0x7fb170106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 --2- 172.21.15.31:0/3222206817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170106a60 0x7fb170106e40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb160009930 tx=0x7fb16002a420 comp rx=0 tx=0).stop 2024-09-16T00:48:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 -- 172.21.15.31:0/3222206817 shutdown_connections 2024-09-16T00:48:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 --2- 172.21.15.31:0/3222206817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb170101400 0x7fb17010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 --2- 172.21.15.31:0/3222206817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb170100a60 0x7fb170100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 --2- 172.21.15.31:0/3222206817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170106a60 0x7fb170106e40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 -- 172.21.15.31:0/3222206817 >> 172.21.15.31:0/3222206817 conn(0x7fb1700fc780 msgr2=0x7fb1700feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 -- 172.21.15.31:0/3222206817 shutdown_connections 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 -- 172.21.15.31:0/3222206817 wait complete. 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.906+0000 7fb175269640 1 Processor -- start 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb175269640 1 -- start start 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb175269640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb170100a60 0x7fb170102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb175269640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170101400 0x7fb170102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb175269640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb170106a60 0x7fb1701030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb175269640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb170110e60 con 0x7fb170101400 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb175269640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb170110ce0 con 0x7fb170106a60 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb175269640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb170110fe0 con 0x7fb170100a60 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170101400 0x7fb170102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170101400 0x7fb170102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36206/0 (socket says 172.21.15.31:36206) 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16e7fc640 1 -- 172.21.15.31:0/4019018886 learned_addr learned my addr 172.21.15.31:0/4019018886 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16f7fe640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb170106a60 0x7fb1701030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16effd640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb170100a60 0x7fb170102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16e7fc640 1 -- 172.21.15.31:0/4019018886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb170100a60 msgr2=0x7fb170102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb170100a60 0x7fb170102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16e7fc640 1 -- 172.21.15.31:0/4019018886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb170106a60 msgr2=0x7fb1701030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb170106a60 0x7fb1701030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16e7fc640 1 -- 172.21.15.31:0/4019018886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1701a68c0 con 0x7fb170101400 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.907+0000 7fb16effd640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb170100a60 0x7fb170102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.908+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170101400 0x7fb170102b70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e9f0 tx=0x7fb16400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.908+0000 7fb157fff640 1 -- 172.21.15.31:0/4019018886 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400cd80 con 0x7fb170101400 2024-09-16T00:48:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.908+0000 7fb175269640 1 -- 172.21.15.31:0/4019018886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1701a6bb0 con 0x7fb170101400 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.908+0000 7fb157fff640 1 -- 172.21.15.31:0/4019018886 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb164010910 con 0x7fb170101400 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.908+0000 7fb157fff640 1 -- 172.21.15.31:0/4019018886 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640188f0 con 0x7fb170101400 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.908+0000 7fb175269640 1 -- 172.21.15.31:0/4019018886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1701a7140 con 0x7fb170101400 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.910+0000 7fb157fff640 1 -- 172.21.15.31:0/4019018886 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb164010ac0 con 0x7fb170101400 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb177642740 1 Processor -- start 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb177642740 1 -- start start 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb177642740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e0e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb157fff640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb1580a3240 0x7fb1580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb177642740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e19140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb177642740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e1ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb177642740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8d742e0 con 0x55f9b8e0e4a0 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb177642740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8d1bce0 con 0x55f9b8e19680 2024-09-16T00:48:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb177642740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8d702a0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e0e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb16effd640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb1580a3240 0x7fb1580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e0e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36220/0 (socket says 172.21.15.31:36220) 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb16f7fe640 1 -- 172.21.15.31:0/2753812436 learned_addr learned my addr 172.21.15.31:0/2753812436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb16f7fe640 1 --2- 172.21.15.31:0/2753812436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e1ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb157fff640 1 -- 172.21.15.31:0/4019018886 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb1640d0130 con 0x7fb170101400 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/2753812436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e19140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.911+0000 7fb16f7fe640 1 -- 172.21.15.31:0/2753812436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0edc0 msgr2=0x55f9b8e19140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.912+0000 7fb16f7fe640 1 --2- 172.21.15.31:0/2753812436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e19140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.912+0000 7fb16f7fe640 1 -- 172.21.15.31:0/2753812436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 msgr2=0x55f9b8e1ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.912+0000 7fb16f7fe640 1 --2- 172.21.15.31:0/2753812436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e1ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.912+0000 7fb16f7fe640 1 -- 172.21.15.31:0/2753812436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b8d00440 con 0x55f9b8e0e4a0 2024-09-16T00:48:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.912+0000 7fb16f7fe640 1 --2- 172.21.15.31:0/2753812436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e0e880 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb168007310 tx=0x7fb168007960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee6b7f65ed6aee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.912+0000 7fb16effd640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb1580a3240 0x7fb1580a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb160000c00 tx=0x7fb160005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.912+0000 7fb1557fa640 1 -- 172.21.15.31:0/2753812436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1680142d0 con 0x55f9b8e0e4a0 2024-09-16T00:48:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.912+0000 7fb1557fa640 1 -- 172.21.15.31:0/2753812436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16800fbf0 con 0x55f9b8e0e4a0 2024-09-16T00:48:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.912+0000 7fb1557fa640 1 -- 172.21.15.31:0/2753812436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1680054e0 con 0x55f9b8e0e4a0 2024-09-16T00:48:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 -- 172.21.15.31:0/2753812436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0e4a0 msgr2=0x55f9b8e0e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 --2- 172.21.15.31:0/2753812436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e0e880 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb168007310 tx=0x7fb168007960 comp rx=0 tx=0).stop 2024-09-16T00:48:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 -- 172.21.15.31:0/2753812436 shutdown_connections 2024-09-16T00:48:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 --2- 172.21.15.31:0/2753812436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e1ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 --2- 172.21.15.31:0/2753812436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e19140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 --2- 172.21.15.31:0/2753812436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e0e880 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 -- 172.21.15.31:0/2753812436 >> 172.21.15.31:0/2753812436 conn(0x55f9b8e0aa30 msgr2=0x55f9b8e0ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 -- 172.21.15.31:0/2753812436 shutdown_connections 2024-09-16T00:48:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 -- 172.21.15.31:0/2753812436 wait complete. 2024-09-16T00:48:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 -- start start 2024-09-16T00:48:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.913+0000 7fb177642740 1 Processor -- start 2024-09-16T00:48:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb177642740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e18fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb177642740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e16270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb177642740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb177642740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8e1c250 con 0x55f9b8e0edc0 2024-09-16T00:48:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb177642740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8d2f050 con 0x55f9b8e19680 2024-09-16T00:48:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb177642740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f9b8e1c3d0 con 0x55f9b8e0e4a0 2024-09-16T00:48:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb177642740 10 client.0 _ll_register_callbacks cb 0x55f9b8d6f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e18fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e16270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e18fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:43266/0 (socket says 172.21.15.31:43266) 2024-09-16T00:48:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16effd640 1 -- 172.21.15.31:0/3509824705 learned_addr learned my addr 172.21.15.31:0/3509824705 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16f7fe640 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16e7fc640 1 -- 172.21.15.31:0/3509824705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 msgr2=0x55f9b8e18fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e18fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16e7fc640 1 -- 172.21.15.31:0/3509824705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 msgr2=0x55f9b8e167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16e7fc640 1 -- 172.21.15.31:0/3509824705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b8ae53c0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e16270 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb1640cdbf0 tx=0x7fb16409bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4cc45540a8e0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb152ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.914+0000 7fb152ff5640 1 -- 172.21.15.31:0/3509824705 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640a5010 con 0x55f9b8e0edc0 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb152ff5640 1 -- 172.21.15.31:0/3509824705 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb1640c0690 con 0x55f9b8e0edc0 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb177642740 1 -- 172.21.15.31:0/3509824705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16800fbf0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb152ff5640 1 -- 172.21.15.31:0/3509824705 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb1640c0120 con 0x55f9b8e0edc0 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb177642740 10 client.16239 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb177642740 1 -- 172.21.15.31:0/3509824705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f9b8e27c00 con 0x55f9b8e0edc0 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb152ff5640 1 -- 172.21.15.31:0/3509824705 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16800fbf0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb152ff5640 1 -- 172.21.15.31:0/3509824705 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb16409b7f0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb177642740 10 client.16239 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb177642740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb177642740 20 client.16239 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb177642740 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9b8e290b0 0x55f9b8e2b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb177642740 4 client.16239 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.915+0000 7fb177642740 1 -- 172.21.15.31:0/3509824705 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b8d81850 con 0x55f9b8e290b0 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.916+0000 7fb16effd640 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9b8e290b0 0x55f9b8e2b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.916+0000 7fb16effd640 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9b8e290b0 0x55f9b8e2b4a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f854db4834c14d52 server_cookie=901a3c20145c209b in_seq=0 out_seq=0 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.916+0000 7fb152ff5640 10 client.16239 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb152ff5640 1 -- 172.21.15.31:0/3509824705 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9b8d81850 con 0x55f9b8e290b0 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb152ff5640 10 client.16239 handle_command_reply: tid=0 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 client.16239 shutdown 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 -- 172.21.15.31:0/3509824705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 msgr2=0x55f9b8e16270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e16270 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb1640cdbf0 tx=0x7fb16409bdf0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 -- 172.21.15.31:0/3509824705 shutdown_connections 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e16270 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9b8e290b0 0x55f9b8e2b4a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 --2- 172.21.15.31:0/3509824705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e18fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb152ff5640 0 client.16239 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 -- 172.21.15.31:0/3509824705 >> 172.21.15.31:0/3509824705 conn(0x55f9b8e0aa30 msgr2=0x55f9b8e10760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.917+0000 7fb177642740 1 -- 172.21.15.31:0/3509824705 shutdown_connections 2024-09-16T00:48:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.918+0000 7fb177642740 1 -- 172.21.15.31:0/3509824705 wait complete. 2024-09-16T00:48:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.918+0000 7fb177642740 20 client.16239 trim_cache size 0 max 16384 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb177642740 1 Processor -- start 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb177642740 1 -- start start 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb177642740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb177642740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e30d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb177642740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e19680 0x55f9b8e31290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb177642740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701aeeb0 con 0x55f9b8e19680 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb177642740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701b1730 con 0x55f9b8e0edc0 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb177642740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701af520 con 0x55f9b8e0e4a0 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb16e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e30d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e19680 0x55f9b8e31290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.951+0000 7fb16e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e30d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:54382/0 (socket says 172.21.15.31:54382) 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb16e7fc640 1 -- 172.21.15.31:0/2152527192 learned_addr learned my addr 172.21.15.31:0/2152527192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb16effd640 1 --2- 172.21.15.31:0/2152527192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb16e7fc640 1 -- 172.21.15.31:0/2152527192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 msgr2=0x55f9b8e254b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/2152527192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e254b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb16e7fc640 1 -- 172.21.15.31:0/2152527192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e19680 msgr2=0x55f9b8e31290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/2152527192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e19680 0x55f9b8e31290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb16e7fc640 1 -- 172.21.15.31:0/2152527192 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b8e18be0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb16f7fe640 1 --2- 172.21.15.31:0/2152527192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e19680 0x55f9b8e31290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/2152527192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e30d50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1640097b0 tx=0x7fb164098770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde3d4d7b9c98288 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb154ff9640 1 -- 172.21.15.31:0/2152527192 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16409c3a0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb154ff9640 1 -- 172.21.15.31:0/2152527192 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16409a6a0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.952+0000 7fb154ff9640 1 -- 172.21.15.31:0/2152527192 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164098370 con 0x55f9b8e0edc0 2024-09-16T00:48:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.953+0000 7fb177642740 1 -- 172.21.15.31:0/2152527192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e0edc0 msgr2=0x55f9b8e30d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.953+0000 7fb177642740 1 --2- 172.21.15.31:0/2152527192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e30d50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1640097b0 tx=0x7fb164098770 comp rx=0 tx=0).stop 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.953+0000 7fb177642740 1 -- 172.21.15.31:0/2152527192 shutdown_connections 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.953+0000 7fb177642740 1 --2- 172.21.15.31:0/2152527192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e19680 0x55f9b8e31290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.953+0000 7fb177642740 1 --2- 172.21.15.31:0/2152527192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e30d50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.953+0000 7fb177642740 1 --2- 172.21.15.31:0/2152527192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.953+0000 7fb177642740 1 -- 172.21.15.31:0/2152527192 >> 172.21.15.31:0/2152527192 conn(0x55f9b8e0aa30 msgr2=0x55f9b8e11900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.953+0000 7fb177642740 1 -- 172.21.15.31:0/2152527192 shutdown_connections 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.953+0000 7fb177642740 1 -- 172.21.15.31:0/2152527192 wait complete. 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb177642740 1 -- start start 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb177642740 1 Processor -- start 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb177642740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e1d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb177642740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e1da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb177642740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e1df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb177642740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701afbf0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb177642740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701b1060 con 0x55f9b8e19680 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb177642740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701b02c0 con 0x55f9b8e0e4a0 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb177642740 10 client.0 _ll_register_callbacks cb 0x7fb16400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb16e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e1da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb16e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e1da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36234/0 (socket says 172.21.15.31:36234) 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb16e7fc640 1 -- 172.21.15.31:0/4152201533 learned_addr learned my addr 172.21.15.31:0/4152201533 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb16f7fe640 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e1df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.954+0000 7fb16effd640 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e1d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb16e7fc640 1 -- 172.21.15.31:0/4152201533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 msgr2=0x55f9b8e1d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e1d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb16e7fc640 1 -- 172.21.15.31:0/4152201533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 msgr2=0x55f9b8e1df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e1df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb16e7fc640 1 -- 172.21.15.31:0/4152201533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b8e92dd0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb16effd640 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e1d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb16f7fe640 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e1df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb16e7fc640 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e1da10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb16409a650 tx=0x7fb16409bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d49dbc122d85d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb152ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb152ff5640 1 -- 172.21.15.31:0/4152201533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640bc480 con 0x55f9b8e0edc0 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb177642740 1 -- 172.21.15.31:0/4152201533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16409a6a0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb152ff5640 1 -- 172.21.15.31:0/4152201533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fb164010430 con 0x55f9b8e0edc0 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb152ff5640 1 -- 172.21.15.31:0/4152201533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb1640af050 con 0x55f9b8e0edc0 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb177642740 10 client.16245 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb177642740 1 -- 172.21.15.31:0/4152201533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f9b8e940c0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.955+0000 7fb152ff5640 1 -- 172.21.15.31:0/4152201533 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16409a6a0 con 0x55f9b8e0edc0 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb152ff5640 1 -- 172.21.15.31:0/4152201533 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb16409b380 con 0x55f9b8e0edc0 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb177642740 10 client.16245 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb177642740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb177642740 20 client.16245 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb177642740 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9b8e945c0 0x55f9b8e969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb177642740 4 client.16245 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb177642740 1 -- 172.21.15.31:0/4152201533 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9b8d7bd60 con 0x55f9b8e945c0 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb16effd640 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9b8e945c0 0x55f9b8e969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb152ff5640 10 client.16245 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.956+0000 7fb16effd640 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9b8e945c0 0x55f9b8e969e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2942b3ef8f5c3f5a server_cookie=c0e7c9465f025698 in_seq=0 out_seq=0 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb152ff5640 1 -- 172.21.15.31:0/4152201533 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f9b8d7bd60 con 0x55f9b8e945c0 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb152ff5640 10 client.16245 handle_command_reply: tid=0 2024-09-16T00:48:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 client.16245 shutdown 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 -- 172.21.15.31:0/4152201533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 msgr2=0x55f9b8e1da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e1da10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb16409a650 tx=0x7fb16409bdf0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 -- 172.21.15.31:0/4152201533 shutdown_connections 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9b8e19680 0x55f9b8e1df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9b8e0edc0 0x55f9b8e1da10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55f9b8e945c0 0x55f9b8e969e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 --2- 172.21.15.31:0/4152201533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9b8e0e4a0 0x55f9b8e1d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb152ff5640 0 client.16245 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 -- 172.21.15.31:0/4152201533 >> 172.21.15.31:0/4152201533 conn(0x55f9b8e0aa30 msgr2=0x55f9b8e17ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:23.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 -- 172.21.15.31:0/4152201533 shutdown_connections 2024-09-16T00:48:23.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 1 -- 172.21.15.31:0/4152201533 wait complete. 2024-09-16T00:48:23.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.958+0000 7fb177642740 20 client.16245 trim_cache size 0 max 16384 2024-09-16T00:48:23.964 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:48:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:48:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:48:23.964 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:48:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 -- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb1580a3240 msgr2=0x7fb1580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb1580a3240 0x7fb1580a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb160000c00 tx=0x7fb160005a20 comp rx=0 tx=0).stop 2024-09-16T00:48:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 -- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170101400 msgr2=0x7fb170102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170101400 0x7fb170102b70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e9f0 tx=0x7fb16400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 -- 172.21.15.31:0/4019018886 shutdown_connections 2024-09-16T00:48:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb170106a60 0x7fb1701030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb170101400 0x7fb170102b70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fb1580a3240 0x7fb1580a5700 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 --2- 172.21.15.31:0/4019018886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb170100a60 0x7fb170102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:23.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 -- 172.21.15.31:0/4019018886 >> 172.21.15.31:0/4019018886 conn(0x7fb1700fc780 msgr2=0x7fb17006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:23.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 -- 172.21.15.31:0/4019018886 shutdown_connections 2024-09-16T00:48:23.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:23.959+0000 7fb175269640 1 -- 172.21.15.31:0/4019018886 wait complete. 2024-09-16T00:48:24.804 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:48:24.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:48:25.077 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:48:25.413 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:25 smithi031 ceph-mon[30956]: pgmap v574: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 748 op/s 2024-09-16T00:48:25.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:25 smithi081 ceph-mon[35539]: pgmap v574: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 748 op/s 2024-09-16T00:48:25.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:25 smithi102 ceph-mon[35364]: pgmap v574: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 748 op/s 2024-09-16T00:48:25.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.625+0000 7fcb5310a640 1 -- 172.21.15.31:0/1542311973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 msgr2=0x7fcb4c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:25.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.625+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1542311973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 0x7fcb4c103110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009c10 tx=0x7fcb4402a6e0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.625+0000 7fcb5310a640 1 -- 172.21.15.31:0/1542311973 shutdown_connections 2024-09-16T00:48:25.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.625+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1542311973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4c1082a0 0x7fcb4c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.625+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1542311973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c1036e0 0x7fcb4c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.625+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1542311973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 0x7fcb4c103110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.625+0000 7fcb5310a640 1 -- 172.21.15.31:0/1542311973 >> 172.21.15.31:0/1542311973 conn(0x7fcb4c0fc7d0 msgr2=0x7fcb4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.625+0000 7fcb5310a640 1 -- 172.21.15.31:0/1542311973 shutdown_connections 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.625+0000 7fcb5310a640 1 -- 172.21.15.31:0/1542311973 wait complete. 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb5310a640 1 Processor -- start 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb5310a640 1 -- start start 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb5310a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 0x7fcb4c1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb5310a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4c1036e0 0x7fcb4c1a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb5310a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c1082a0 0x7fcb4c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb5310a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c112050 con 0x7fcb4c1036e0 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb5310a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c111ed0 con 0x7fcb4c102d30 2024-09-16T00:48:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb5310a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c1121d0 con 0x7fcb4c1082a0 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb4bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4c1036e0 0x7fcb4c1a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb50ea6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 0x7fcb4c1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb4bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4c1036e0 0x7fcb4c1a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36258/0 (socket says 172.21.15.31:36258) 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb4bfff640 1 -- 172.21.15.31:0/1903354925 learned_addr learned my addr 172.21.15.31:0/1903354925 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb50ea6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 0x7fcb4c1a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:54408/0 (socket says 172.21.15.31:54408) 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb516a7640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c1082a0 0x7fcb4c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb4bfff640 1 -- 172.21.15.31:0/1903354925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c1082a0 msgr2=0x7fcb4c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb4bfff640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c1082a0 0x7fcb4c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb4bfff640 1 -- 172.21.15.31:0/1903354925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 msgr2=0x7fcb4c1a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb4bfff640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 0x7fcb4c1a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb4bfff640 1 -- 172.21.15.31:0/1903354925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c19ad10 con 0x7fcb4c1036e0 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.626+0000 7fcb50ea6640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 0x7fcb4c1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:25.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.627+0000 7fcb4bfff640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4c1036e0 0x7fcb4c1a0960 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000d950 tx=0x7fcb4000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.627+0000 7fcb49ffb640 1 -- 172.21.15.31:0/1903354925 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40017070 con 0x7fcb4c1036e0 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.627+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c19b000 con 0x7fcb4c1036e0 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.627+0000 7fcb49ffb640 1 -- 172.21.15.31:0/1903354925 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb40009c30 con 0x7fcb4c1036e0 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.627+0000 7fcb49ffb640 1 -- 172.21.15.31:0/1903354925 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400047a0 con 0x7fcb4c1036e0 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.627+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c1a72f0 con 0x7fcb4c1036e0 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.628+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4c06a850 con 0x7fcb4c1036e0 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.629+0000 7fcb49ffb640 1 -- 172.21.15.31:0/1903354925 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcb40004020 con 0x7fcb4c1036e0 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.630+0000 7fcb49ffb640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb340a3240 0x7fcb340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.630+0000 7fcb49ffb640 1 -- 172.21.15.31:0/1903354925 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fcb400cfd30 con 0x7fcb4c1036e0 2024-09-16T00:48:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.630+0000 7fcb50ea6640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb340a3240 0x7fcb340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:25.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.631+0000 7fcb50ea6640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb340a3240 0x7fcb340a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcb44006010 tx=0x7fcb44000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:25.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.634+0000 7fcb49ffb640 1 -- 172.21.15.31:0/1903354925 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb40020020 con 0x7fcb4c1036e0 2024-09-16T00:48:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.853+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb4c103110 con 0x7fcb4c1036e0 2024-09-16T00:48:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.854+0000 7fcb49ffb640 1 -- 172.21.15.31:0/1903354925 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fcb4009c6a0 con 0x7fcb4c1036e0 2024-09-16T00:48:25.855 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:48:25.855 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:48:25.855 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:48:25.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb340a3240 msgr2=0x7fcb340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:25.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb340a3240 0x7fcb340a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcb44006010 tx=0x7fcb44000f40 comp rx=0 tx=0).stop 2024-09-16T00:48:25.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4c1036e0 msgr2=0x7fcb4c1a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:25.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4c1036e0 0x7fcb4c1a0960 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000d950 tx=0x7fcb4000dd50 comp rx=0 tx=0).stop 2024-09-16T00:48:25.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 shutdown_connections 2024-09-16T00:48:25.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcb340a3240 0x7fcb340a5700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c1082a0 0x7fcb4c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4c1036e0 0x7fcb4c1a0960 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 --2- 172.21.15.31:0/1903354925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb4c102d30 0x7fcb4c1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:25.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.856+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 >> 172.21.15.31:0/1903354925 conn(0x7fcb4c0fc7d0 msgr2=0x7fcb4c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:25.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.857+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 shutdown_connections 2024-09-16T00:48:25.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:25.857+0000 7fcb5310a640 1 -- 172.21.15.31:0/1903354925 wait complete. 2024-09-16T00:48:26.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:26 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1903354925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:48:26.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:26 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1903354925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:48:26.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:26 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1903354925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:48:26.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:48:26.922 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:48:27.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:27 smithi031 ceph-mon[30956]: pgmap v575: 129 pgs: 129 active+clean; 339 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 585 op/s 2024-09-16T00:48:27.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:27 smithi081 ceph-mon[35539]: pgmap v575: 129 pgs: 129 active+clean; 339 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 585 op/s 2024-09-16T00:48:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.526+0000 7f71e1d9b640 1 -- 172.21.15.31:0/2139330093 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc030d30 con 0x7f71e4110190 2024-09-16T00:48:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.527+0000 7f71e8a33640 1 -- 172.21.15.31:0/2139330093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e4110190 msgr2=0x7f71e406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.527+0000 7f71e8a33640 1 --2- 172.21.15.31:0/2139330093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e4110190 0x7f71e406e4f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009ca0 tx=0x7f71dc02a8b0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.527+0000 7f71e8a33640 1 -- 172.21.15.31:0/2139330093 shutdown_connections 2024-09-16T00:48:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.527+0000 7f71e8a33640 1 --2- 172.21.15.31:0/2139330093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71e411b140 0x7f71e411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.527+0000 7f71e8a33640 1 --2- 172.21.15.31:0/2139330093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e406ea30 0x7f71e406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.527+0000 7f71e8a33640 1 --2- 172.21.15.31:0/2139330093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e4110190 0x7f71e406e4f0 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009ca0 tx=0x7f71dc02a8b0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.527+0000 7f71e8a33640 1 -- 172.21.15.31:0/2139330093 >> 172.21.15.31:0/2139330093 conn(0x7f71e4073180 msgr2=0x7f71e4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.527+0000 7f71e8a33640 1 -- 172.21.15.31:0/2139330093 shutdown_connections 2024-09-16T00:48:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.528+0000 7f71e8a33640 1 -- 172.21.15.31:0/2139330093 wait complete. 2024-09-16T00:48:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.530+0000 7f71e8a33640 1 Processor -- start 2024-09-16T00:48:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.531+0000 7f71e8a33640 1 -- start start 2024-09-16T00:48:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.531+0000 7f71e8a33640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e406ea30 0x7f71e41a7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.531+0000 7f71e8a33640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e411b140 0x7f71e41a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.531+0000 7f71e8a33640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71e41abb40 0x7f71e41abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.531+0000 7f71e8a33640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4120be0 con 0x7f71e41abb40 2024-09-16T00:48:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.531+0000 7f71e8a33640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4120a60 con 0x7f71e406ea30 2024-09-16T00:48:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.531+0000 7f71e8a33640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4120d60 con 0x7f71e411b140 2024-09-16T00:48:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.532+0000 7f71e2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e406ea30 0x7f71e41a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.532+0000 7f71e2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e406ea30 0x7f71e41a7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34570/0 (socket says 172.21.15.31:34570) 2024-09-16T00:48:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.532+0000 7f71e2d9d640 1 -- 172.21.15.31:0/1961851182 learned_addr learned my addr 172.21.15.31:0/1961851182 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.532+0000 7f71e259c640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e411b140 0x7f71e41a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.532+0000 7f71e359e640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71e41abb40 0x7f71e41abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.533+0000 7f71e359e640 1 -- 172.21.15.31:0/1961851182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e411b140 msgr2=0x7f71e41a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.533+0000 7f71e359e640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e411b140 0x7f71e41a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.533+0000 7f71e359e640 1 -- 172.21.15.31:0/1961851182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e406ea30 msgr2=0x7f71e41a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.533+0000 7f71e359e640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e406ea30 0x7f71e41a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.533+0000 7f71e359e640 1 -- 172.21.15.31:0/1961851182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e41ac600 con 0x7f71e41abb40 2024-09-16T00:48:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.533+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e406ea30 0x7f71e41a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.534+0000 7f71e359e640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71e41abb40 0x7f71e41abff0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f71e406b4a0 tx=0x7f71d400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.535+0000 7f71cbfff640 1 -- 172.21.15.31:0/1961851182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4005220 con 0x7f71e41abb40 2024-09-16T00:48:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.535+0000 7f71cbfff640 1 -- 172.21.15.31:0/1961851182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d4004c80 con 0x7f71e41abb40 2024-09-16T00:48:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.535+0000 7f71cbfff640 1 -- 172.21.15.31:0/1961851182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4007e50 con 0x7f71e41abb40 2024-09-16T00:48:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.535+0000 7f71e8a33640 1 -- 172.21.15.31:0/1961851182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e41ac8f0 con 0x7f71e41abb40 2024-09-16T00:48:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.535+0000 7f71e8a33640 1 -- 172.21.15.31:0/1961851182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e41b4080 con 0x7f71e41abb40 2024-09-16T00:48:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.539+0000 7f71eb60d740 1 Processor -- start 2024-09-16T00:48:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.539+0000 7f71cbfff640 1 -- 172.21.15.31:0/1961851182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f71d40053c0 con 0x7f71e41abb40 2024-09-16T00:48:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71eb60d740 1 -- start start 2024-09-16T00:48:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9f0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71eb60d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56068d9562f0 con 0x56068d9f1d20 2024-09-16T00:48:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71eb60d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56068d8fdcf0 con 0x56068d9fb550 2024-09-16T00:48:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71eb60d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56068d9522b0 con 0x56068d9f07b0 2024-09-16T00:48:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71e259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71e259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47700/0 (socket says 172.21.15.31:47700) 2024-09-16T00:48:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71e259c640 1 -- 172.21.15.31:0/1827975575 learned_addr learned my addr 172.21.15.31:0/1827975575 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.540+0000 7f71e259c640 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71e259c640 1 -- 172.21.15.31:0/1827975575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 msgr2=0x56068d9f0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71e259c640 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9f0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71e259c640 1 -- 172.21.15.31:0/1827975575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 msgr2=0x56068d9f0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71e259c640 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9f0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71e259c640 1 -- 172.21.15.31:0/1827975575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068d8e23a0 con 0x56068d9fb550 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9f0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71cbfff640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71cc0a34a0 0x7f71cc0a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71e259c640 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fd940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71d80118c0 tx=0x7f71d800a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6af214ff4aba44a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71cbfff640 1 -- 172.21.15.31:0/1961851182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f71d40cf5b0 con 0x7f71e41abb40 2024-09-16T00:48:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71cc0a34a0 0x7f71cc0a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71c97fa640 1 -- 172.21.15.31:0/1827975575 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004020 con 0x56068d9fb550 2024-09-16T00:48:27.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.541+0000 7f71c97fa640 1 -- 172.21.15.31:0/1827975575 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d800de30 con 0x56068d9fb550 2024-09-16T00:48:27.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71c97fa640 1 -- 172.21.15.31:0/1827975575 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8003cc0 con 0x56068d9fb550 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71eb60d740 1 -- 172.21.15.31:0/1827975575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 msgr2=0x56068d9fd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fd940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71d80118c0 tx=0x7f71d800a400 comp rx=0 tx=0).stop 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71cc0a34a0 0x7f71cc0a5960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x56068d9edca0 tx=0x7f71dc000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71eb60d740 1 -- 172.21.15.31:0/1827975575 shutdown_connections 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fd940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9f0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1827975575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71eb60d740 1 -- 172.21.15.31:0/1827975575 >> 172.21.15.31:0/1827975575 conn(0x56068d9b9310 msgr2=0x56068d9ece40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71eb60d740 1 -- 172.21.15.31:0/1827975575 shutdown_connections 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.542+0000 7f71eb60d740 1 -- 172.21.15.31:0/1827975575 wait complete. 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71eb60d740 1 -- start start 2024-09-16T00:48:27.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71eb60d740 1 Processor -- start 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9fae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9f8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71eb60d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56068d9fe2a0 con 0x56068d9f1d20 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71eb60d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56068d911060 con 0x56068d9fb550 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71eb60d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56068d9fe420 con 0x56068d9f07b0 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71eb60d740 10 client.0 _ll_register_callbacks cb 0x56068d6648c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71e359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71e359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34574/0 (socket says 172.21.15.31:34574) 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71e359e640 1 -- 172.21.15.31:0/1483761349 learned_addr learned my addr 172.21.15.31:0/1483761349 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71e259c640 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.543+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71e359e640 1 -- 172.21.15.31:0/1483761349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 msgr2=0x56068d9fae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71e359e640 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9fae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71e359e640 1 -- 172.21.15.31:0/1483761349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 msgr2=0x56068d9f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71e359e640 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71e359e640 1 -- 172.21.15.31:0/1483761349 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068d6c73c0 con 0x56068d9fb550 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71e359e640 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9f8660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71d409b800 tx=0x7f71d409b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c651fadf4719113 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71c6ff5640 1 -- 172.21.15.31:0/1483761349 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d40cf040 con 0x56068d9fb550 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71eb60d740 1 -- 172.21.15.31:0/1483761349 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d800de30 con 0x56068d9fb550 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71c6ff5640 1 -- 172.21.15.31:0/1483761349 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f71d40cf1e0 con 0x56068d9fb550 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71c6ff5640 1 -- 172.21.15.31:0/1483761349 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f71d40b9050 con 0x56068d9fb550 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71eb60d740 10 client.25940 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71eb60d740 1 -- 172.21.15.31:0/1483761349 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56068da09d10 con 0x56068d9fb550 2024-09-16T00:48:27.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.544+0000 7f71c6ff5640 1 -- 172.21.15.31:0/1483761349 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d800de30 con 0x56068d9fb550 2024-09-16T00:48:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71c6ff5640 1 -- 172.21.15.31:0/1483761349 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f71d40afce0 con 0x56068d9fb550 2024-09-16T00:48:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71eb60d740 10 client.25940 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71eb60d740 10 client.25940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:48:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71eb60d740 20 client.25940 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56068da0b120 0x56068da0d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71eb60d740 4 client.25940 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:48:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71eb60d740 1 -- 172.21.15.31:0/1483761349 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56068d963860 con 0x56068da0b120 2024-09-16T00:48:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56068da0b120 0x56068da0d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71c6ff5640 10 client.25940 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.545+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56068da0b120 0x56068da0d580 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dca688fb0065706 server_cookie=d617ab65ff902994 in_seq=0 out_seq=0 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71c6ff5640 1 -- 172.21.15.31:0/1483761349 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56068d963860 con 0x56068da0b120 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71c6ff5640 10 client.25940 handle_command_reply: tid=0 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71eb60d740 1 client.25940 shutdown 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71eb60d740 1 -- 172.21.15.31:0/1483761349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 msgr2=0x56068d9f8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9f8660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71d409b800 tx=0x7f71d409b9b0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71eb60d740 1 -- 172.21.15.31:0/1483761349 shutdown_connections 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9f8660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56068da0b120 0x56068da0d580 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71c6ff5640 0 client.25940 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1483761349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.546+0000 7f71eb60d740 1 -- 172.21.15.31:0/1483761349 >> 172.21.15.31:0/1483761349 conn(0x56068d9b9310 msgr2=0x56068d9f2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:27.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.547+0000 7f71eb60d740 1 -- 172.21.15.31:0/1483761349 shutdown_connections 2024-09-16T00:48:27.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.547+0000 7f71eb60d740 1 -- 172.21.15.31:0/1483761349 wait complete. 2024-09-16T00:48:27.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.547+0000 7f71eb60d740 20 client.25940 trim_cache size 0 max 16384 2024-09-16T00:48:27.555 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:27 smithi102 ceph-mon[35364]: pgmap v575: 129 pgs: 129 active+clean; 339 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 585 op/s 2024-09-16T00:48:27.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.560+0000 7f71eb60d740 1 Processor -- start 2024-09-16T00:48:27.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.560+0000 7f71eb60d740 1 -- start start 2024-09-16T00:48:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.560+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9fa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.560+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9fc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.560+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.560+0000 7f71eb60d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71d000db80 con 0x56068d9f1d20 2024-09-16T00:48:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.560+0000 7f71eb60d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0010400 con 0x56068d9fb550 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71eb60d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71d000e1f0 con 0x56068d9f07b0 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9fc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9fc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47716/0 (socket says 172.21.15.31:47716) 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e259c640 1 -- 172.21.15.31:0/1761256565 learned_addr learned my addr 172.21.15.31:0/1761256565 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e359e640 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e359e640 1 -- 172.21.15.31:0/1761256565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 msgr2=0x56068d9fa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e359e640 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9fa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e359e640 1 -- 172.21.15.31:0/1761256565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 msgr2=0x56068d9fc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e359e640 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9fc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e359e640 1 -- 172.21.15.31:0/1761256565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068d9faa90 con 0x56068d9fb550 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e259c640 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71e359e640 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fce80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f71d4098240 tx=0x7f71d409b240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e649fd80ee2008a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71c8ff9640 1 -- 172.21.15.31:0/1761256565 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d401d040 con 0x56068d9fb550 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71c8ff9640 1 -- 172.21.15.31:0/1761256565 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d4086070 con 0x56068d9fb550 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.561+0000 7f71c8ff9640 1 -- 172.21.15.31:0/1761256565 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d409acc0 con 0x56068d9fb550 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 -- 172.21.15.31:0/1761256565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 msgr2=0x56068d9fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fce80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f71d4098240 tx=0x7f71d409b240 comp rx=0 tx=0).stop 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 -- 172.21.15.31:0/1761256565 shutdown_connections 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9fb550 0x56068d9fce80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9f1d20 0x56068d9fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 --2- 172.21.15.31:0/1761256565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068d9fa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 -- 172.21.15.31:0/1761256565 >> 172.21.15.31:0/1761256565 conn(0x56068d9b9310 msgr2=0x56068d9f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 -- 172.21.15.31:0/1761256565 shutdown_connections 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 -- 172.21.15.31:0/1761256565 wait complete. 2024-09-16T00:48:27.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 -- start start 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.562+0000 7f71eb60d740 1 Processor -- start 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068da03ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9f1d20 0x56068da128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71eb60d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9fb550 0x56068da13050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71eb60d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71d000e8c0 con 0x56068d9fb550 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71eb60d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f71d000fd30 con 0x56068d9f1d20 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71eb60d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71d000ef90 con 0x56068d9f07b0 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71eb60d740 10 client.0 _ll_register_callbacks cb 0x7f71d4097450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e2d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068da03ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9f1d20 0x56068da128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e2d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068da03ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:39706/0 (socket says 172.21.15.31:39706) 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e2d9d640 1 -- 172.21.15.31:0/671074107 learned_addr learned my addr 172.21.15.31:0/671074107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9fb550 0x56068da13050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e2d9d640 1 -- 172.21.15.31:0/671074107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9f1d20 msgr2=0x56068da128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9f1d20 0x56068da128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e2d9d640 1 -- 172.21.15.31:0/671074107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9fb550 msgr2=0x56068da13050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9fb550 0x56068da13050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e2d9d640 1 -- 172.21.15.31:0/671074107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068da01a70 con 0x56068d9f07b0 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e259c640 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9f1d20 0x56068da128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e359e640 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9fb550 0x56068da13050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71e2d9d640 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068da03ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005e30 tx=0x7f71dc0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f15eb9dc3e8158f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.563+0000 7f71c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71c6ff5640 1 -- 172.21.15.31:0/671074107 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc030d40 con 0x56068d9f07b0 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71c6ff5640 1 -- 172.21.15.31:0/671074107 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f71dc031880 con 0x56068d9f07b0 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71eb60d740 1 -- 172.21.15.31:0/671074107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d4086070 con 0x56068d9f07b0 2024-09-16T00:48:27.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71c6ff5640 1 -- 172.21.15.31:0/671074107 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f71dc031a80 con 0x56068d9f07b0 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71eb60d740 10 client.25864 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71eb60d740 1 -- 172.21.15.31:0/671074107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56068da75fa0 con 0x56068d9f07b0 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71c6ff5640 1 -- 172.21.15.31:0/671074107 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d4086070 con 0x56068d9f07b0 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71c6ff5640 1 -- 172.21.15.31:0/671074107 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f71dc044040 con 0x56068d9f07b0 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71eb60d740 10 client.25864 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71eb60d740 10 client.25864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71eb60d740 20 client.25864 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71eb60d740 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56068da0b120 0x56068da786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71eb60d740 4 client.25864 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71eb60d740 1 -- 172.21.15.31:0/671074107 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56068d95dd70 con 0x56068da0b120 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.564+0000 7f71e259c640 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56068da0b120 0x56068da786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.565+0000 7f71e259c640 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56068da0b120 0x56068da786e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb5898c4fde735b server_cookie=2c7e2934eb8eb942 in_seq=0 out_seq=0 2024-09-16T00:48:27.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.565+0000 7f71c6ff5640 10 client.25864 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:27.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.566+0000 7f71c6ff5640 1 -- 172.21.15.31:0/671074107 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56068d95dd70 con 0x56068da0b120 2024-09-16T00:48:27.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.566+0000 7f71c6ff5640 10 client.25864 handle_command_reply: tid=0 2024-09-16T00:48:27.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.566+0000 7f71eb60d740 1 client.25864 shutdown 2024-09-16T00:48:27.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.566+0000 7f71eb60d740 1 -- 172.21.15.31:0/671074107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 msgr2=0x56068da03ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:27.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.566+0000 7f71eb60d740 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068da03ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005e30 tx=0x7f71dc0095c0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71eb60d740 1 -- 172.21.15.31:0/671074107 shutdown_connections 2024-09-16T00:48:27.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71eb60d740 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56068d9fb550 0x56068da13050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71eb60d740 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56068d9f1d20 0x56068da128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71eb60d740 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x56068da0b120 0x56068da786e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71eb60d740 1 --2- 172.21.15.31:0/671074107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56068d9f07b0 0x56068da03ff0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71eb60d740 1 -- 172.21.15.31:0/671074107 >> 172.21.15.31:0/671074107 conn(0x56068d9b9310 msgr2=0x56068d9fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71c6ff5640 0 client.25864 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71eb60d740 1 -- 172.21.15.31:0/671074107 shutdown_connections 2024-09-16T00:48:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71eb60d740 1 -- 172.21.15.31:0/671074107 wait complete. 2024-09-16T00:48:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71eb60d740 20 client.25864 trim_cache size 0 max 16384 2024-09-16T00:48:27.576 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:48:27.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 -- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71cc0a34a0 msgr2=0x7f71cc0a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:27.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71cc0a34a0 0x7f71cc0a5960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x56068d9edca0 tx=0x7f71dc000f40 comp rx=0 tx=0).stop 2024-09-16T00:48:27.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 -- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71e41abb40 msgr2=0x7f71e41abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:27.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71e41abb40 0x7f71e41abff0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f71e406b4a0 tx=0x7f71d400cad0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 -- 172.21.15.31:0/1961851182 shutdown_connections 2024-09-16T00:48:27.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71e41abb40 0x7f71e41abff0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71cc0a34a0 0x7f71cc0a5960 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e411b140 0x7f71e41a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 --2- 172.21.15.31:0/1961851182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e406ea30 0x7f71e41a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:27.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 -- 172.21.15.31:0/1961851182 >> 172.21.15.31:0/1961851182 conn(0x7f71e4073180 msgr2=0x7f71e4119130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:27.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.567+0000 7f71e8a33640 1 -- 172.21.15.31:0/1961851182 shutdown_connections 2024-09-16T00:48:27.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:27.568+0000 7f71e8a33640 1 -- 172.21.15.31:0/1961851182 wait complete. 2024-09-16T00:48:29.318 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:48:29.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:48:29.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:29 smithi081 ceph-mon[35539]: pgmap v576: 129 pgs: 129 active+clean; 319 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 641 op/s 2024-09-16T00:48:29.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:29 smithi031 ceph-mon[30956]: pgmap v576: 129 pgs: 129 active+clean; 319 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 641 op/s 2024-09-16T00:48:29.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:29 smithi102 ceph-mon[35364]: pgmap v576: 129 pgs: 129 active+clean; 319 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 641 op/s 2024-09-16T00:48:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.140+0000 7f6778ef6640 1 -- 172.21.15.31:0/2376244006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 msgr2=0x7f6774106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.140+0000 7f6778ef6640 1 --2- 172.21.15.31:0/2376244006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 0x7f6774106e60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6764009930 tx=0x7f676402a420 comp rx=0 tx=0).stop 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.140+0000 7f6778ef6640 1 -- 172.21.15.31:0/2376244006 shutdown_connections 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.140+0000 7f6778ef6640 1 --2- 172.21.15.31:0/2376244006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6774101420 0x7f677410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.140+0000 7f6778ef6640 1 --2- 172.21.15.31:0/2376244006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6774100a80 0x7f6774100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.140+0000 7f6778ef6640 1 --2- 172.21.15.31:0/2376244006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 0x7f6774106e60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.140+0000 7f6778ef6640 1 -- 172.21.15.31:0/2376244006 >> 172.21.15.31:0/2376244006 conn(0x7f67740fc7a0 msgr2=0x7f67740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.140+0000 7f6778ef6640 1 -- 172.21.15.31:0/2376244006 shutdown_connections 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.140+0000 7f6778ef6640 1 -- 172.21.15.31:0/2376244006 wait complete. 2024-09-16T00:48:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6778ef6640 1 Processor -- start 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6778ef6640 1 -- start start 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6778ef6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6774100a80 0x7f677419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6778ef6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6774101420 0x7f677419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6778ef6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 0x7f677419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6778ef6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6774110e80 con 0x7f6774106a80 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6778ef6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6774110d00 con 0x7f6774100a80 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6778ef6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6774111000 con 0x7f6774101420 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6772d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 0x7f677419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f677259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6774100a80 0x7f677419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.141+0000 7f6772d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 0x7f677419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47738/0 (socket says 172.21.15.31:47738) 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f6772d9d640 1 -- 172.21.15.31:0/1641698430 learned_addr learned my addr 172.21.15.31:0/1641698430 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f677259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6774100a80 0x7f677419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34600/0 (socket says 172.21.15.31:34600) 2024-09-16T00:48:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f676bfff640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6774101420 0x7f677419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f6772d9d640 1 -- 172.21.15.31:0/1641698430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6774101420 msgr2=0x7f677419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f6772d9d640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6774101420 0x7f677419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f6772d9d640 1 -- 172.21.15.31:0/1641698430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6774100a80 msgr2=0x7f677419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f6772d9d640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6774100a80 0x7f677419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f6772d9d640 1 -- 172.21.15.31:0/1641698430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677419f660 con 0x7f6774106a80 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f677259c640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6774100a80 0x7f677419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f6772d9d640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 0x7f677419ef80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f676c00eb10 tx=0x7f676c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f676b7fe640 1 -- 172.21.15.31:0/1641698430 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c004b20 con 0x7f6774106a80 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f6778ef6640 1 -- 172.21.15.31:0/1641698430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677419f950 con 0x7f6774106a80 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f676b7fe640 1 -- 172.21.15.31:0/1641698430 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676c004cc0 con 0x7f6774106a80 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f676b7fe640 1 -- 172.21.15.31:0/1641698430 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c010760 con 0x7f6774106a80 2024-09-16T00:48:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.142+0000 7f6778ef6640 1 -- 172.21.15.31:0/1641698430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a7230 con 0x7f6774106a80 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.145+0000 7f676b7fe640 1 -- 172.21.15.31:0/1641698430 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f676c005020 con 0x7f6774106a80 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.145+0000 7f677b2cf740 1 Processor -- start 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.145+0000 7f677b2cf740 1 -- start start 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.145+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575eeb60 0x5589575eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.145+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575f9d40 0x5589575fc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f677b2cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558957554280 con 0x5589575ef480 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f677b2cf740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5589574fbb10 con 0x5589575f9d40 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f677b2cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558957550240 con 0x5589575eeb60 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f677259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f677259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47748/0 (socket says 172.21.15.31:47748) 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f677259c640 1 -- 172.21.15.31:0/1581046819 learned_addr learned my addr 172.21.15.31:0/1581046819 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676bfff640 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575f9d40 0x5589575fc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676b7fe640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f67600a3240 0x7f67600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676bfff640 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575eeb60 0x5589575eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f6772d9d640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f67600a3240 0x7f67600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676bfff640 1 -- 172.21.15.31:0/1581046819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575eeb60 msgr2=0x5589575eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676bfff640 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575eeb60 0x5589575eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676b7fe640 1 -- 172.21.15.31:0/1641698430 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f676c0d0210 con 0x7f6774106a80 2024-09-16T00:48:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676bfff640 1 -- 172.21.15.31:0/1581046819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 msgr2=0x5589575f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676bfff640 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676bfff640 1 -- 172.21.15.31:0/1581046819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589575081f0 con 0x5589575f9d40 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f677259c640 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f676bfff640 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575f9d40 0x5589575fc130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f675c005d00 tx=0x7f675c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6527d2c51e5f6bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.146+0000 7f6772d9d640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f67600a3240 0x7f67600a5700 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f676c0049d0 tx=0x7f676c01bb30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f6768ff9640 1 -- 172.21.15.31:0/1581046819 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c012ca0 con 0x5589575f9d40 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f6768ff9640 1 -- 172.21.15.31:0/1581046819 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c012e40 con 0x5589575f9d40 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f6768ff9640 1 -- 172.21.15.31:0/1581046819 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c004a00 con 0x5589575f9d40 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f677b2cf740 1 -- 172.21.15.31:0/1581046819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575f9d40 msgr2=0x5589575fc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575f9d40 0x5589575fc130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f675c005d00 tx=0x7f675c005c90 comp rx=0 tx=0).stop 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f677b2cf740 1 -- 172.21.15.31:0/1581046819 shutdown_connections 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575f9d40 0x5589575fc130 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1581046819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575eeb60 0x5589575eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f677b2cf740 1 -- 172.21.15.31:0/1581046819 >> 172.21.15.31:0/1581046819 conn(0x5589575eb0f0 msgr2=0x5589575eb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.147+0000 7f677b2cf740 1 -- 172.21.15.31:0/1581046819 shutdown_connections 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 1 -- 172.21.15.31:0/1581046819 wait complete. 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 1 -- start start 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 1 Processor -- start 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x5589575f7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5589575fc910 con 0x5589575ef480 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55895750ef10 con 0x5589575eeb60 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5589575fca90 con 0x5589575f9d40 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677b2cf740 10 client.0 _ll_register_callbacks cb 0x5589574d7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f676bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f676bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34624/0 (socket says 172.21.15.31:34624) 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f677259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47758/0 (socket says 172.21.15.31:47758) 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.148+0000 7f676bfff640 1 -- 172.21.15.31:0/1164570617 learned_addr learned my addr 172.21.15.31:0/1164570617 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f676bfff640 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x5589575f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f677259c640 1 -- 172.21.15.31:0/1164570617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 msgr2=0x5589575f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f677259c640 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x5589575f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f677259c640 1 -- 172.21.15.31:0/1164570617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 msgr2=0x5589575f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f677259c640 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f677259c640 1 -- 172.21.15.31:0/1164570617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589572c56a0 con 0x5589575ef480 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f676bfff640 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f677259c640 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f71e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6764006010 tx=0x7f6764031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550f6aa3fa081d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f6759ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f6759ffb640 1 -- 172.21.15.31:0/1164570617 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764035040 con 0x5589575ef480 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f6759ffb640 1 -- 172.21.15.31:0/1164570617 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f6764004440 con 0x5589575ef480 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f677b2cf740 1 -- 172.21.15.31:0/1164570617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589575617f0 con 0x5589575ef480 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f6759ffb640 1 -- 172.21.15.31:0/1164570617 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6764005050 con 0x5589575ef480 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f677b2cf740 10 client.16278 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.149+0000 7f677b2cf740 1 -- 172.21.15.31:0/1164570617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558957608590 con 0x5589575ef480 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f6759ffb640 1 -- 172.21.15.31:0/1164570617 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5589575617f0 con 0x5589575ef480 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f6759ffb640 1 -- 172.21.15.31:0/1164570617 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f67640035f0 con 0x5589575ef480 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f677b2cf740 10 client.16278 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f677b2cf740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f677b2cf740 20 client.16278 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558957609a50 0x55895760be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f677b2cf740 4 client.16278 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f677b2cf740 1 -- 172.21.15.31:0/1164570617 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895755bd00 con 0x558957609a50 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f676bfff640 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558957609a50 0x55895760be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f676bfff640 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558957609a50 0x55895760be40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83bb48d2eaa181b server_cookie=51d9ae200d080fce in_seq=0 out_seq=0 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.150+0000 7f6759ffb640 10 client.16278 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.151+0000 7f6759ffb640 1 -- 172.21.15.31:0/1164570617 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55895755bd00 con 0x558957609a50 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.151+0000 7f6759ffb640 10 client.16278 handle_command_reply: tid=0 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.151+0000 7f677b2cf740 1 client.16278 shutdown 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 -- 172.21.15.31:0/1164570617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 msgr2=0x5589575f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f71e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6764006010 tx=0x7f6764031850 comp rx=0 tx=0).stop 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 -- 172.21.15.31:0/1164570617 shutdown_connections 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558957609a50 0x55895760be40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x5589575f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x5589575f71e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1164570617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 -- 172.21.15.31:0/1164570617 >> 172.21.15.31:0/1164570617 conn(0x5589575eb0f0 msgr2=0x5589575f0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f6759ffb640 0 client.16278 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 -- 172.21.15.31:0/1164570617 shutdown_connections 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 1 -- 172.21.15.31:0/1164570617 wait complete. 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.152+0000 7f677b2cf740 20 client.16278 trim_cache size 0 max 16384 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677b2cf740 1 Processor -- start 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677b2cf740 1 -- start start 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x558957611710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x558957611c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677b2cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67741aefd0 con 0x5589575ef480 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677b2cf740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b1850 con 0x5589575eeb60 2024-09-16T00:48:30.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677b2cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f67741af640 con 0x5589575f9d40 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x558957611710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f676bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x558957611710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47772/0 (socket says 172.21.15.31:47772) 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f676bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34638/0 (socket says 172.21.15.31:34638) 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.170+0000 7f677259c640 1 -- 172.21.15.31:0/2282352117 learned_addr learned my addr 172.21.15.31:0/2282352117 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f676bfff640 1 --2- 172.21.15.31:0/2282352117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x558957611c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f677259c640 1 -- 172.21.15.31:0/2282352117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 msgr2=0x558957611c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f677259c640 1 --2- 172.21.15.31:0/2282352117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x558957611c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f677259c640 1 -- 172.21.15.31:0/2282352117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 msgr2=0x5589575f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f677259c640 1 --2- 172.21.15.31:0/2282352117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f677259c640 1 -- 172.21.15.31:0/2282352117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589575f9570 con 0x5589575ef480 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f677259c640 1 --2- 172.21.15.31:0/2282352117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x558957611710 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f676402a940 tx=0x7f676402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f609b1be14e88b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:30.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f675bfff640 1 -- 172.21.15.31:0/2282352117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764035030 con 0x5589575ef480 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f675bfff640 1 -- 172.21.15.31:0/2282352117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6764041070 con 0x5589575ef480 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.171+0000 7f675bfff640 1 -- 172.21.15.31:0/2282352117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67640318c0 con 0x5589575ef480 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 -- 172.21.15.31:0/2282352117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 msgr2=0x558957611710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 --2- 172.21.15.31:0/2282352117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x558957611710 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f676402a940 tx=0x7f676402a420 comp rx=0 tx=0).stop 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 -- 172.21.15.31:0/2282352117 shutdown_connections 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 --2- 172.21.15.31:0/2282352117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x558957611c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 --2- 172.21.15.31:0/2282352117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575ef480 0x558957611710 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 --2- 172.21.15.31:0/2282352117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575eeb60 0x5589575f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 -- 172.21.15.31:0/2282352117 >> 172.21.15.31:0/2282352117 conn(0x5589575eb0f0 msgr2=0x5589575f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 -- 172.21.15.31:0/2282352117 shutdown_connections 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 -- 172.21.15.31:0/2282352117 wait complete. 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 -- start start 2024-09-16T00:48:30.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.172+0000 7f677b2cf740 1 Processor -- start 2024-09-16T00:48:30.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575eeb60 0x5589575fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575ef480 0x5589575fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f677b2cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x5589575fe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f677b2cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67741afd10 con 0x5589575eeb60 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f677b2cf740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b1180 con 0x5589575ef480 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f677b2cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b03e0 con 0x5589575f9d40 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f677b2cf740 10 client.0 _ll_register_callbacks cb 0x7f6764005dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575eeb60 0x5589575fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575eeb60 0x5589575fdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47776/0 (socket says 172.21.15.31:47776) 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 -- 172.21.15.31:0/1593941539 learned_addr learned my addr 172.21.15.31:0/1593941539 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f677259c640 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575ef480 0x5589575fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x5589575fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 -- 172.21.15.31:0/1593941539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 msgr2=0x5589575fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:30.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x5589575fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 -- 172.21.15.31:0/1593941539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575ef480 msgr2=0x5589575fe0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575ef480 0x5589575fe0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 -- 172.21.15.31:0/1593941539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558957673880 con 0x5589575eeb60 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f677259c640 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575ef480 0x5589575fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f676bfff640 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575eeb60 0x5589575fdbb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f675c012720 tx=0x7f675c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cfdbd075929b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.173+0000 7f6759ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f6759ffb640 1 -- 172.21.15.31:0/1593941539 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c031040 con 0x5589575eeb60 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f6759ffb640 1 -- 172.21.15.31:0/1593941539 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f675c010880 con 0x5589575eeb60 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677b2cf740 1 -- 172.21.15.31:0/1593941539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6764041070 con 0x5589575eeb60 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f6759ffb640 1 -- 172.21.15.31:0/1593941539 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f675c03f050 con 0x5589575eeb60 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677b2cf740 10 client.16284 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677b2cf740 1 -- 172.21.15.31:0/1593941539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558957674b60 con 0x5589575eeb60 2024-09-16T00:48:30.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f6759ffb640 1 -- 172.21.15.31:0/1593941539 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6764041070 con 0x5589575eeb60 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f6759ffb640 1 -- 172.21.15.31:0/1593941539 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f675c02d040 con 0x5589575eeb60 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677b2cf740 10 client.16284 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677b2cf740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677b2cf740 20 client.16284 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558957675030 0x558957677420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677b2cf740 4 client.16284 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677b2cf740 1 -- 172.21.15.31:0/1593941539 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5589574d8110 con 0x558957675030 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.174+0000 7f677259c640 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558957675030 0x558957677420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.175+0000 7f6759ffb640 10 client.16284 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.175+0000 7f677259c640 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558957675030 0x558957677420 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d0fafcecda74a2 server_cookie=9aaa292548407f88 in_seq=0 out_seq=0 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f6759ffb640 1 -- 172.21.15.31:0/1593941539 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5589574d8110 con 0x558957675030 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f6759ffb640 10 client.16284 handle_command_reply: tid=0 2024-09-16T00:48:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 client.16284 shutdown 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 -- 172.21.15.31:0/1593941539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575eeb60 msgr2=0x5589575fdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575eeb60 0x5589575fdbb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f675c012720 tx=0x7f675c00b5b0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 -- 172.21.15.31:0/1593941539 shutdown_connections 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x558957675030 0x558957677420 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589575f9d40 0x5589575fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589575ef480 0x5589575fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 --2- 172.21.15.31:0/1593941539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589575eeb60 0x5589575fdbb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 -- 172.21.15.31:0/1593941539 >> 172.21.15.31:0/1593941539 conn(0x5589575eb0f0 msgr2=0x5589575ea270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f6759ffb640 0 client.16284 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 -- 172.21.15.31:0/1593941539 shutdown_connections 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 1 -- 172.21.15.31:0/1593941539 wait complete. 2024-09-16T00:48:30.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.177+0000 7f677b2cf740 20 client.16284 trim_cache size 0 max 16384 2024-09-16T00:48:30.193 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:48:30.193 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-16T00:48:30.193 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "8e99e3d6-8752-4d41-8467-f502ff5e95a5", 2024-09-16T00:48:30.193 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-16T00:48:30.193 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 -- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f67600a3240 msgr2=0x7f67600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f67600a3240 0x7f67600a5700 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f676c0049d0 tx=0x7f676c01bb30 comp rx=0 tx=0).stop 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 -- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 msgr2=0x7f677419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 0x7f677419ef80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f676c00eb10 tx=0x7f676c00c2d0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 -- 172.21.15.31:0/1641698430 shutdown_connections 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774106a80 0x7f677419ef80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f67600a3240 0x7f67600a5700 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6774101420 0x7f677419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 --2- 172.21.15.31:0/1641698430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6774100a80 0x7f677419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 -- 172.21.15.31:0/1641698430 >> 172.21.15.31:0/1641698430 conn(0x7f67740fc7a0 msgr2=0x7f677406a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:30.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 -- 172.21.15.31:0/1641698430 shutdown_connections 2024-09-16T00:48:30.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:30.178+0000 7f6778ef6640 1 -- 172.21.15.31:0/1641698430 wait complete. 2024-09-16T00:48:30.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:48:31.150 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:48:31.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:31 smithi081 ceph-mon[35539]: pgmap v577: 129 pgs: 129 active+clean; 295 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 658 op/s 2024-09-16T00:48:31.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:31 smithi081 ceph-mon[35539]: scrub queued for path: / 2024-09-16T00:48:31.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:31 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:48:31.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:31 smithi081 ceph-mon[35539]: scrub summary: active paths [/] 2024-09-16T00:48:31.536 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:31 smithi031 ceph-mon[30956]: pgmap v577: 129 pgs: 129 active+clean; 295 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 658 op/s 2024-09-16T00:48:31.536 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:31 smithi031 ceph-mon[30956]: scrub queued for path: / 2024-09-16T00:48:31.536 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:31 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:48:31.536 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:31 smithi031 ceph-mon[30956]: scrub summary: active paths [/] 2024-09-16T00:48:31.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:31 smithi102 ceph-mon[35364]: pgmap v577: 129 pgs: 129 active+clean; 295 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 658 op/s 2024-09-16T00:48:31.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:31 smithi102 ceph-mon[35364]: scrub queued for path: / 2024-09-16T00:48:31.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:31 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:48:31.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:31 smithi102 ceph-mon[35364]: scrub summary: active paths [/] 2024-09-16T00:48:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.168+0000 7f711c83d640 1 -- 172.21.15.31:0/2872991274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7118102df0 msgr2=0x7f71181031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.168+0000 7f711c83d640 1 --2- 172.21.15.31:0/2872991274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7118102df0 0x7f71181031d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7108009900 tx=0x7f710802a410 comp rx=0 tx=0).stop 2024-09-16T00:48:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.168+0000 7f711c83d640 1 -- 172.21.15.31:0/2872991274 shutdown_connections 2024-09-16T00:48:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.168+0000 7f711c83d640 1 --2- 172.21.15.31:0/2872991274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f711810c2f0 0x7f711810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.168+0000 7f711c83d640 1 --2- 172.21.15.31:0/2872991274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71181037a0 0x7f711810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.168+0000 7f711c83d640 1 --2- 172.21.15.31:0/2872991274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7118102df0 0x7f71181031d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.168+0000 7f711c83d640 1 -- 172.21.15.31:0/2872991274 >> 172.21.15.31:0/2872991274 conn(0x7f71180fc7d0 msgr2=0x7f71180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.168+0000 7f711c83d640 1 -- 172.21.15.31:0/2872991274 shutdown_connections 2024-09-16T00:48:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.168+0000 7f711c83d640 1 -- 172.21.15.31:0/2872991274 wait complete. 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.169+0000 7f711c83d640 1 Processor -- start 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.169+0000 7f711c83d640 1 -- start start 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.169+0000 7f711c83d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7118102df0 0x7f7118198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.169+0000 7f711c83d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71181037a0 0x7f7118198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.169+0000 7f711c83d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f711810c2f0 0x7f711819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.169+0000 7f711c83d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71181120f0 con 0x7f711810c2f0 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.169+0000 7f711c83d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7118111f70 con 0x7f7118102df0 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.169+0000 7f711c83d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7118112270 con 0x7f71181037a0 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f711659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7118102df0 0x7f7118198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f7116d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f711810c2f0 0x7f711819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f711659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7118102df0 0x7f7118198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34670/0 (socket says 172.21.15.31:34670) 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f711659c640 1 -- 172.21.15.31:0/401633396 learned_addr learned my addr 172.21.15.31:0/401633396 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f710ffff640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71181037a0 0x7f7118198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f7116d9d640 1 -- 172.21.15.31:0/401633396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71181037a0 msgr2=0x7f7118198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f7116d9d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71181037a0 0x7f7118198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f7116d9d640 1 -- 172.21.15.31:0/401633396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7118102df0 msgr2=0x7f7118198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f7116d9d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7118102df0 0x7f7118198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f7116d9d640 1 -- 172.21.15.31:0/401633396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711819d5c0 con 0x7f711810c2f0 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f711659c640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7118102df0 0x7f7118198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f7116d9d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f711810c2f0 0x7f711819cee0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7110009fd0 tx=0x7f711000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f710f7fe640 1 -- 172.21.15.31:0/401633396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711000ce20 con 0x7f711810c2f0 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f711c83d640 1 -- 172.21.15.31:0/401633396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711819d8b0 con 0x7f711810c2f0 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f710f7fe640 1 -- 172.21.15.31:0/401633396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7110005020 con 0x7f711810c2f0 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f710f7fe640 1 -- 172.21.15.31:0/401633396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110010720 con 0x7f711810c2f0 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.170+0000 7f711c83d640 1 -- 172.21.15.31:0/401633396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a5190 con 0x7f711810c2f0 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.173+0000 7f710f7fe640 1 -- 172.21.15.31:0/401633396 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7110005540 con 0x7f711810c2f0 2024-09-16T00:48:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.173+0000 7f711ec16740 1 Processor -- start 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.173+0000 7f711ec16740 1 -- start start 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.173+0000 7f711ec16740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c6522a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.173+0000 7f711ec16740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65239550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.173+0000 7f711ec16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 0x557c6523be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.173+0000 7f711ec16740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557c651902e0 con 0x557c6522a3a0 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.173+0000 7f711ec16740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557c65137ce0 con 0x557c6522acc0 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.173+0000 7f711ec16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557c6518c2a0 con 0x557c65239a90 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c6522a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c6522a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47808/0 (socket says 172.21.15.31:47808) 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 -- 172.21.15.31:0/108542455 learned_addr learned my addr 172.21.15.31:0/108542455 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:32.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 0x557c6523be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710f7fe640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71040a3240 0x7f71040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 -- 172.21.15.31:0/108542455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 msgr2=0x557c6523be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 0x557c6523be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 -- 172.21.15.31:0/108542455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 msgr2=0x557c65239550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f711659c640 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65239550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65239550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 -- 172.21.15.31:0/108542455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6511c440 con 0x557c6522a3a0 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f711659c640 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65239550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f7116d9d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71040a3240 0x7f71040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710ffff640 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c6522a780 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f71000136e0 tx=0x7f7100009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b1c8f8ed3cecc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.174+0000 7f710f7fe640 1 -- 172.21.15.31:0/401633396 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f71100d0350 con 0x7f711810c2f0 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f710cff9640 1 -- 172.21.15.31:0/108542455 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100013d20 con 0x557c6522a3a0 2024-09-16T00:48:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f710cff9640 1 -- 172.21.15.31:0/108542455 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71000130d0 con 0x557c6522a3a0 2024-09-16T00:48:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f710cff9640 1 -- 172.21.15.31:0/108542455 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710000fc40 con 0x557c6522a3a0 2024-09-16T00:48:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f711ec16740 1 -- 172.21.15.31:0/108542455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 msgr2=0x557c6522a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f7116d9d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71040a3240 0x7f71040a5700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f711009c5f0 tx=0x7f7110066ff0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f711ec16740 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c6522a780 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f71000136e0 tx=0x7f7100009620 comp rx=0 tx=0).stop 2024-09-16T00:48:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f711ec16740 1 -- 172.21.15.31:0/108542455 shutdown_connections 2024-09-16T00:48:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f711ec16740 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 0x557c6523be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f711ec16740 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65239550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f711ec16740 1 --2- 172.21.15.31:0/108542455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c6522a780 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f711ec16740 1 -- 172.21.15.31:0/108542455 >> 172.21.15.31:0/108542455 conn(0x557c65226a30 msgr2=0x557c65226e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.175+0000 7f711ec16740 1 -- 172.21.15.31:0/108542455 shutdown_connections 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 1 -- 172.21.15.31:0/108542455 wait complete. 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 1 -- start start 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 1 Processor -- start 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c65239420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c652366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 0x557c65236c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557c6523c660 con 0x557c6522a3a0 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557c6514b050 con 0x557c6522acc0 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557c6523c7e0 con 0x557c65239a90 2024-09-16T00:48:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711ec16740 10 client.0 _ll_register_callbacks cb 0x557c6518b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f710ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c65239420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f710ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c65239420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47814/0 (socket says 172.21.15.31:47814) 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f711659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c652366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.176+0000 7f710ffff640 1 -- 172.21.15.31:0/1989664172 learned_addr learned my addr 172.21.15.31:0/1989664172 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f710ffff640 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 0x557c65236c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f710ffff640 1 -- 172.21.15.31:0/1989664172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 msgr2=0x557c65236c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f710ffff640 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 0x557c65236c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f710ffff640 1 -- 172.21.15.31:0/1989664172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 msgr2=0x557c652366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f710ffff640 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c652366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f710ffff640 1 -- 172.21.15.31:0/1989664172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c64f013c0 con 0x557c6522a3a0 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f711659c640 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c652366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f710ffff640 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c65239420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7100005780 tx=0x7f71000056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f18d3a8d17d5d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f70fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:48:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f70fdffb640 1 -- 172.21.15.31:0/1989664172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100013280 con 0x557c6522a3a0 2024-09-16T00:48:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f70fdffb640 1 -- 172.21.15.31:0/1989664172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f7100011ab0 con 0x557c6522a3a0 2024-09-16T00:48:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f711ec16740 1 -- 172.21.15.31:0/1989664172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71000130d0 con 0x557c6522a3a0 2024-09-16T00:48:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f70fdffb640 1 -- 172.21.15.31:0/1989664172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7100020410 con 0x557c6522a3a0 2024-09-16T00:48:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f711ec16740 10 client.16296 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f711ec16740 1 -- 172.21.15.31:0/1989664172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557c65248000 con 0x557c6522a3a0 2024-09-16T00:48:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.177+0000 7f70fdffb640 1 -- 172.21.15.31:0/1989664172 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71000130d0 con 0x557c6522a3a0 2024-09-16T00:48:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f70fdffb640 1 -- 172.21.15.31:0/1989664172 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f7100010630 con 0x557c6522a3a0 2024-09-16T00:48:32.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f711ec16740 10 client.16296 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f711ec16740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f711ec16740 20 client.16296 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f711ec16740 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557c65249350 0x557c6524b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f711ec16740 4 client.16296 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f711ec16740 1 -- 172.21.15.31:0/1989664172 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c6519d850 con 0x557c65249350 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f711659c640 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557c65249350 0x557c6524b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f711659c640 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557c65249350 0x557c6524b740 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1fe6336a08b91d server_cookie=514ae900d4e6c7d in_seq=0 out_seq=0 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.178+0000 7f70fdffb640 10 client.16296 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f70fdffb640 1 -- 172.21.15.31:0/1989664172 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c6519d850 con 0x557c65249350 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f70fdffb640 10 client.16296 handle_command_reply: tid=0 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 client.16296 shutdown 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 -- 172.21.15.31:0/1989664172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 msgr2=0x557c65239420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c65239420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7100005780 tx=0x7f71000056f0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 -- 172.21.15.31:0/1989664172 shutdown_connections 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557c65249350 0x557c6524b740 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c65239a90 0x557c65236c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c652366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 --2- 172.21.15.31:0/1989664172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c6522a3a0 0x557c65239420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f70fdffb640 0 client.16296 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 -- 172.21.15.31:0/1989664172 >> 172.21.15.31:0/1989664172 conn(0x557c65226a30 msgr2=0x557c6522c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 -- 172.21.15.31:0/1989664172 shutdown_connections 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 1 -- 172.21.15.31:0/1989664172 wait complete. 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.181+0000 7f711ec16740 20 client.16296 trim_cache size 0 max 16384 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.199+0000 7f711ec16740 1 Processor -- start 2024-09-16T00:48:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.199+0000 7f711ec16740 1 -- start start 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.199+0000 7f711ec16740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522a3a0 0x557c65245900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.199+0000 7f711ec16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522acc0 0x557c65235f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.199+0000 7f711ec16740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65231d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f711ec16740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71181aaae0 con 0x557c65239a90 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f711ec16740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f71181ad360 con 0x557c6522a3a0 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f711ec16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71181ab150 con 0x557c6522acc0 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f710ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522a3a0 0x557c65245900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f710ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65231d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f711659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522acc0 0x557c65235f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f711659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522acc0 0x557c65235f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:39810/0 (socket says 172.21.15.31:39810) 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f710ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522a3a0 0x557c65245900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:34690/0 (socket says 172.21.15.31:34690) 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f711659c640 1 -- 172.21.15.31:0/2797961900 learned_addr learned my addr 172.21.15.31:0/2797961900 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f710ffff640 1 -- 172.21.15.31:0/2797961900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522acc0 msgr2=0x557c65235f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f710ffff640 1 --2- 172.21.15.31:0/2797961900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522acc0 0x557c65235f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f710ffff640 1 -- 172.21.15.31:0/2797961900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 msgr2=0x557c65231d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f710ffff640 1 --2- 172.21.15.31:0/2797961900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65231d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f710ffff640 1 -- 172.21.15.31:0/2797961900 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6522aaf0 con 0x557c6522a3a0 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f710ffff640 1 --2- 172.21.15.31:0/2797961900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522a3a0 0x557c65245900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7100001eb0 tx=0x7f71000056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b7c4338e886fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.200+0000 7f70fffff640 1 -- 172.21.15.31:0/2797961900 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100009b80 con 0x557c6522a3a0 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f70fffff640 1 -- 172.21.15.31:0/2797961900 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7100010c70 con 0x557c6522a3a0 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f70fffff640 1 -- 172.21.15.31:0/2797961900 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100011b10 con 0x557c6522a3a0 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f711ec16740 1 -- 172.21.15.31:0/2797961900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522a3a0 msgr2=0x557c65245900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:32.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f711ec16740 1 --2- 172.21.15.31:0/2797961900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522a3a0 0x557c65245900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7100001eb0 tx=0x7f71000056f0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f711ec16740 1 -- 172.21.15.31:0/2797961900 shutdown_connections 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f711ec16740 1 --2- 172.21.15.31:0/2797961900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65231d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f711ec16740 1 --2- 172.21.15.31:0/2797961900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522acc0 0x557c65235f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f711ec16740 1 --2- 172.21.15.31:0/2797961900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522a3a0 0x557c65245900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f711ec16740 1 -- 172.21.15.31:0/2797961900 >> 172.21.15.31:0/2797961900 conn(0x557c65226a30 msgr2=0x557c65226e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f711ec16740 1 -- 172.21.15.31:0/2797961900 shutdown_connections 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.201+0000 7f711ec16740 1 -- 172.21.15.31:0/2797961900 wait complete. 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711ec16740 1 -- start start 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711ec16740 1 Processor -- start 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711ec16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522a3a0 0x557c65241a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711ec16740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65241fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711ec16740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65232d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711ec16740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71181ab820 con 0x557c65239a90 2024-09-16T00:48:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711ec16740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f71181acc90 con 0x557c6522acc0 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711ec16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71181abef0 con 0x557c6522a3a0 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711ec16740 10 client.0 _ll_register_callbacks cb 0x7f7100009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f710ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65232d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f710ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65232d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47834/0 (socket says 172.21.15.31:47834) 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f710ffff640 1 -- 172.21.15.31:0/1298900139 learned_addr learned my addr 172.21.15.31:0/1298900139 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f711659c640 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65241fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.202+0000 7f710ffff640 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522a3a0 0x557c65241a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f710ffff640 1 -- 172.21.15.31:0/1298900139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522a3a0 msgr2=0x557c65241a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f710ffff640 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522a3a0 0x557c65241a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f710ffff640 1 -- 172.21.15.31:0/1298900139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 msgr2=0x557c65241fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f710ffff640 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65241fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f710ffff640 1 -- 172.21.15.31:0/1298900139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c65233270 con 0x557c65239a90 2024-09-16T00:48:32.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f711659c640 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65241fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f710ffff640 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65232d00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f710000fd70 tx=0x7f7100009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1909173ae5bed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f70fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f70fdffb640 1 -- 172.21.15.31:0/1298900139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100009b80 con 0x557c65239a90 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f70fdffb640 1 -- 172.21.15.31:0/1298900139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f71000050c0 con 0x557c65239a90 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f711ec16740 1 -- 172.21.15.31:0/1298900139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7100010c70 con 0x557c65239a90 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f70fdffb640 1 -- 172.21.15.31:0/1298900139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f71000052c0 con 0x557c65239a90 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f711ec16740 10 client.16302 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.203+0000 7f711ec16740 1 -- 172.21.15.31:0/1298900139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557c652b3550 con 0x557c65239a90 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f70fdffb640 1 -- 172.21.15.31:0/1298900139 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7100010c70 con 0x557c65239a90 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f70fdffb640 1 -- 172.21.15.31:0/1298900139 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f7100043020 con 0x557c65239a90 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f711ec16740 10 client.16302 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f711ec16740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f711ec16740 20 client.16302 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:32.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f711ec16740 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557c652b3b30 0x557c652b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f711ec16740 4 client.16302 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f711ec16740 1 -- 172.21.15.31:0/1298900139 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c65197d60 con 0x557c652b3b30 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f711659c640 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557c652b3b30 0x557c652b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f711659c640 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557c652b3b30 0x557c652b5f50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d4f41e1a7a02175 server_cookie=f0f35bcfae5781c5 in_seq=0 out_seq=0 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.204+0000 7f70fdffb640 10 client.16302 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.206+0000 7f70fdffb640 1 -- 172.21.15.31:0/1298900139 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x557c65197d60 con 0x557c652b3b30 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.206+0000 7f70fdffb640 10 client.16302 handle_command_reply: tid=0 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.206+0000 7f711ec16740 1 client.16302 shutdown 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 -- 172.21.15.31:0/1298900139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 msgr2=0x557c65232d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:32.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65232d00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f710000fd70 tx=0x7f7100009260 comp rx=0 tx=0).stop 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 -- 172.21.15.31:0/1298900139 shutdown_connections 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c65239a90 0x557c65232d00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c6522acc0 0x557c65241fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x557c652b3b30 0x557c652b5f50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f70fdffb640 0 client.16302 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 --2- 172.21.15.31:0/1298900139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c6522a3a0 0x557c65241a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 -- 172.21.15.31:0/1298900139 >> 172.21.15.31:0/1298900139 conn(0x557c65226a30 msgr2=0x557c65231d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 -- 172.21.15.31:0/1298900139 shutdown_connections 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 1 -- 172.21.15.31:0/1298900139 wait complete. 2024-09-16T00:48:32.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711ec16740 20 client.16302 trim_cache size 0 max 16384 2024-09-16T00:48:32.222 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:48:32.222 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (2529 inodes in the stack)", 2024-09-16T00:48:32.222 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-16T00:48:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "8e99e3d6-8752-4d41-8467-f502ff5e95a5": { 2024-09-16T00:48:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-16T00:48:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "8e99e3d6-8752-4d41-8467-f502ff5e95a5", 2024-09-16T00:48:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-16T00:48:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:48:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:48:32.223 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:48:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711c83d640 1 -- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71040a3240 msgr2=0x7f71040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711c83d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71040a3240 0x7f71040a5700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f711009c5f0 tx=0x7f7110066ff0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711c83d640 1 -- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f711810c2f0 msgr2=0x7f711819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.207+0000 7f711c83d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f711810c2f0 0x7f711819cee0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7110009fd0 tx=0x7f711000ef10 comp rx=0 tx=0).stop 2024-09-16T00:48:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.208+0000 7f711c83d640 1 -- 172.21.15.31:0/401633396 shutdown_connections 2024-09-16T00:48:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.208+0000 7f711c83d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f711810c2f0 0x7f711819cee0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.208+0000 7f711c83d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f71040a3240 0x7f71040a5700 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.208+0000 7f711c83d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71181037a0 0x7f7118198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.208+0000 7f711c83d640 1 --2- 172.21.15.31:0/401633396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7118102df0 0x7f7118198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.208+0000 7f711c83d640 1 -- 172.21.15.31:0/401633396 >> 172.21.15.31:0/401633396 conn(0x7f71180fc7d0 msgr2=0x7f71180fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.208+0000 7f711c83d640 1 -- 172.21.15.31:0/401633396 shutdown_connections 2024-09-16T00:48:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:32.208+0000 7f711c83d640 1 -- 172.21.15.31:0/401633396 wait complete. 2024-09-16T00:48:32.967 INFO:tasks.cephfs.filesystem:scrub status for tag:8e99e3d6-8752-4d41-8467-f502ff5e95a5 - {'path': '/', 'tag': '8e99e3d6-8752-4d41-8467-f502ff5e95a5', 'options': 'recursive,force'} 2024-09-16T00:48:33.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:33 smithi031 ceph-mon[30956]: pgmap v578: 129 pgs: 129 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 682 op/s 2024-09-16T00:48:33.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:33 smithi081 ceph-mon[35539]: pgmap v578: 129 pgs: 129 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 682 op/s 2024-09-16T00:48:33.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:33 smithi102 ceph-mon[35364]: pgmap v578: 129 pgs: 129 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 682 op/s 2024-09-16T00:48:35.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:35 smithi081 ceph-mon[35539]: pgmap v579: 129 pgs: 129 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 889 op/s 2024-09-16T00:48:35.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:35 smithi102 ceph-mon[35364]: pgmap v579: 129 pgs: 129 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 889 op/s 2024-09-16T00:48:35.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:35 smithi031 ceph-mon[30956]: pgmap v579: 129 pgs: 129 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 889 op/s 2024-09-16T00:48:36.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:36 smithi081 ceph-mon[35539]: scrub summary: idle+waiting paths [/] 2024-09-16T00:48:36.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:36 smithi102 ceph-mon[35364]: scrub summary: idle+waiting paths [/] 2024-09-16T00:48:36.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:36 smithi031 ceph-mon[30956]: scrub summary: idle+waiting paths [/] 2024-09-16T00:48:37.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:37 smithi081 ceph-mon[35539]: pgmap v580: 129 pgs: 129 active+clean; 267 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 798 op/s 2024-09-16T00:48:37.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:37 smithi102 ceph-mon[35364]: pgmap v580: 129 pgs: 129 active+clean; 267 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 798 op/s 2024-09-16T00:48:37.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:37 smithi031 ceph-mon[30956]: pgmap v580: 129 pgs: 129 active+clean; 267 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 798 op/s 2024-09-16T00:48:39.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:39 smithi081 ceph-mon[35539]: pgmap v581: 129 pgs: 129 active+clean; 263 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 925 op/s 2024-09-16T00:48:39.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:39 smithi102 ceph-mon[35364]: pgmap v581: 129 pgs: 129 active+clean; 263 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 925 op/s 2024-09-16T00:48:39.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:39 smithi031 ceph-mon[30956]: pgmap v581: 129 pgs: 129 active+clean; 263 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 925 op/s 2024-09-16T00:48:40.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:40 smithi031 ceph-mon[30956]: scrub summary: idle 2024-09-16T00:48:40.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:40 smithi081 ceph-mon[35539]: scrub summary: idle 2024-09-16T00:48:40.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:40 smithi102 ceph-mon[35364]: scrub summary: idle 2024-09-16T00:48:41.451 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:41 smithi102 ceph-mon[35364]: pgmap v582: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 953 op/s 2024-09-16T00:48:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:41 smithi081 ceph-mon[35539]: pgmap v582: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 953 op/s 2024-09-16T00:48:41.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:41 smithi031 ceph-mon[30956]: pgmap v582: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 953 op/s 2024-09-16T00:48:42.483 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-16T00:48:42.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T00:48:42.908 DEBUG:teuthology.parallel:result is None 2024-09-16T00:48:42.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:48:42.937 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:48:42.938 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-16T00:48:42.948 INFO:tasks.quiescer:joining Quiescers 2024-09-16T00:48:42.948 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-16T00:48:42.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:48:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:48:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:43 smithi081 ceph-mon[35539]: pgmap v583: 129 pgs: 129 active+clean; 232 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 943 op/s 2024-09-16T00:48:43.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:43 smithi102 ceph-mon[35364]: pgmap v583: 129 pgs: 129 active+clean; 232 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 943 op/s 2024-09-16T00:48:43.618 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:43 smithi031 ceph-mon[30956]: pgmap v583: 129 pgs: 129 active+clean; 232 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 943 op/s 2024-09-16T00:48:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.847+0000 7fa0a05c3640 1 -- 172.21.15.31:0/3709525339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 msgr2=0x7fa098103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.847+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/3709525339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 0x7fa098103120 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa094009900 tx=0x7fa09402a480 comp rx=0 tx=0).stop 2024-09-16T00:48:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.847+0000 7fa0a05c3640 1 -- 172.21.15.31:0/3709525339 shutdown_connections 2024-09-16T00:48:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.847+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/3709525339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa09810c170 0x7fa09810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.847+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/3709525339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa098103660 0x7fa09810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.847+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/3709525339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 0x7fa098103120 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.847+0000 7fa0a05c3640 1 -- 172.21.15.31:0/3709525339 >> 172.21.15.31:0/3709525339 conn(0x7fa0980fc7a0 msgr2=0x7fa0980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.847+0000 7fa0a05c3640 1 -- 172.21.15.31:0/3709525339 shutdown_connections 2024-09-16T00:48:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.847+0000 7fa0a05c3640 1 -- 172.21.15.31:0/3709525339 wait complete. 2024-09-16T00:48:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.848+0000 7fa0a05c3640 1 Processor -- start 2024-09-16T00:48:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.848+0000 7fa0a05c3640 1 -- start start 2024-09-16T00:48:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.848+0000 7fa0a05c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 0x7fa098198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa0a05c3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa098103660 0x7fa0981989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa0a05c3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa09810c170 0x7fa09819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa0a05c3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa098111f70 con 0x7fa098102d40 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa0a05c3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa098111df0 con 0x7fa098103660 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa0a05c3640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981120f0 con 0x7fa09810c170 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09e35f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 0x7fa098198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09e35f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 0x7fa098198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45148/0 (socket says 172.21.15.31:45148) 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09e35f640 1 -- 172.21.15.31:0/4028786057 learned_addr learned my addr 172.21.15.31:0/4028786057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09eb60640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa09810c170 0x7fa09819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09db5e640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa098103660 0x7fa0981989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09e35f640 1 -- 172.21.15.31:0/4028786057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa09810c170 msgr2=0x7fa09819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09e35f640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa09810c170 0x7fa09819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09e35f640 1 -- 172.21.15.31:0/4028786057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa098103660 msgr2=0x7fa0981989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09e35f640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa098103660 0x7fa0981989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09e35f640 1 -- 172.21.15.31:0/4028786057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09819d420 con 0x7fa098102d40 2024-09-16T00:48:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa09e35f640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 0x7fa098198470 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa094006010 tx=0x7fa0940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa08f7fe640 1 -- 172.21.15.31:0/4028786057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09403f030 con 0x7fa098102d40 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09819d6b0 con 0x7fa098102d40 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa08f7fe640 1 -- 172.21.15.31:0/4028786057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa094003ce0 con 0x7fa098102d40 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.849+0000 7fa08f7fe640 1 -- 172.21.15.31:0/4028786057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094035070 con 0x7fa098102d40 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.850+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0981a4ff0 con 0x7fa098102d40 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.851+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa098103120 con 0x7fa098102d40 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.852+0000 7fa08f7fe640 1 -- 172.21.15.31:0/4028786057 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa094042030 con 0x7fa098102d40 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.852+0000 7fa08f7fe640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0840a3240 0x7fa0840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.852+0000 7fa08f7fe640 1 -- 172.21.15.31:0/4028786057 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa0940f0220 con 0x7fa098102d40 2024-09-16T00:48:43.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.852+0000 7fa09db5e640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0840a3240 0x7fa0840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:43.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.853+0000 7fa09db5e640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0840a3240 0x7fa0840a5700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa09810d4d0 tx=0x7fa088005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:43.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:43.857+0000 7fa08f7fe640 1 -- 172.21.15.31:0/4028786057 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0940bcc50 con 0x7fa098102d40 2024-09-16T00:48:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.072+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0981071e0 con 0x7fa098102d40 2024-09-16T00:48:44.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.073+0000 7fa08f7fe640 1 -- 172.21.15.31:0/4028786057 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7fa0940bc550 con 0x7fa098102d40 2024-09-16T00:48:44.073 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:48:44.074 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:48:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0840a3240 msgr2=0x7fa0840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0840a3240 0x7fa0840a5700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa09810d4d0 tx=0x7fa088005f90 comp rx=0 tx=0).stop 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 msgr2=0x7fa098198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 0x7fa098198470 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa094006010 tx=0x7fa0940042f0 comp rx=0 tx=0).stop 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 shutdown_connections 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa0840a3240 0x7fa0840a5700 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa09810c170 0x7fa09819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa098103660 0x7fa0981989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 --2- 172.21.15.31:0/4028786057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa098102d40 0x7fa098198470 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 >> 172.21.15.31:0/4028786057 conn(0x7fa0980fc7a0 msgr2=0x7fa0980fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:44.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 shutdown_connections 2024-09-16T00:48:44.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:44.075+0000 7fa0a05c3640 1 -- 172.21.15.31:0/4028786057 wait complete. 2024-09-16T00:48:44.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:44 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4028786057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:48:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:44 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4028786057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:48:44.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:44 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4028786057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:48:45.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:45 smithi031 ceph-mon[30956]: pgmap v584: 129 pgs: 129 active+clean; 198 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 1.02k op/s 2024-09-16T00:48:45.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:45 smithi081 ceph-mon[35539]: pgmap v584: 129 pgs: 129 active+clean; 198 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 1.02k op/s 2024-09-16T00:48:45.507 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --cancel --all' 2024-09-16T00:48:45.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --cancel --all 2024-09-16T00:48:45.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:45 smithi102 ceph-mon[35364]: pgmap v584: 129 pgs: 129 active+clean; 198 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 1.02k op/s 2024-09-16T00:48:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 -- 172.21.15.31:0/590642093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d010e480 msgr2=0x7fe3d0110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/590642093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d010e480 0x7fe3d0110870 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009ca0 tx=0x7fe3c802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 -- 172.21.15.31:0/590642093 shutdown_connections 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/590642093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d010e480 0x7fe3d0110870 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/590642093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3d0101280 0x7fe3d010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/590642093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3d01008d0 0x7fe3d0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 -- 172.21.15.31:0/590642093 >> 172.21.15.31:0/590642093 conn(0x7fe3d00fc7a0 msgr2=0x7fe3d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 -- 172.21.15.31:0/590642093 shutdown_connections 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 -- 172.21.15.31:0/590642093 wait complete. 2024-09-16T00:48:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 Processor -- start 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.278+0000 7fe3d77a7640 1 -- start start 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d77a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d01008d0 0x7fe3d019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d77a7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3d0101280 0x7fe3d019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d77a7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3d010e480 0x7fe3d019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d77a7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0114180 con 0x7fe3d01008d0 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d77a7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0114000 con 0x7fe3d0101280 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d77a7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0114300 con 0x7fe3d010e480 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5543640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d01008d0 0x7fe3d019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5543640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d01008d0 0x7fe3d019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45166/0 (socket says 172.21.15.31:45166) 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5543640 1 -- 172.21.15.31:0/2491821041 learned_addr learned my addr 172.21.15.31:0/2491821041 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3d010e480 0x7fe3d019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3d0101280 0x7fe3d019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5543640 1 -- 172.21.15.31:0/2491821041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3d010e480 msgr2=0x7fe3d019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5543640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3d010e480 0x7fe3d019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5543640 1 -- 172.21.15.31:0/2491821041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3d0101280 msgr2=0x7fe3d019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5543640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3d0101280 0x7fe3d019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5543640 1 -- 172.21.15.31:0/2491821041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d019f650 con 0x7fe3d01008d0 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3d010e480 0x7fe3d019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3d0101280 0x7fe3d019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.279+0000 7fe3d5543640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d01008d0 0x7fe3d019a6a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00d950 tx=0x7fe3cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.280+0000 7fe3c67fc640 1 -- 172.21.15.31:0/2491821041 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc017070 con 0x7fe3d01008d0 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.280+0000 7fe3c67fc640 1 -- 172.21.15.31:0/2491821041 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc009c30 con 0x7fe3d01008d0 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.280+0000 7fe3d77a7640 1 -- 172.21.15.31:0/2491821041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d019f940 con 0x7fe3d01008d0 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.280+0000 7fe3c67fc640 1 -- 172.21.15.31:0/2491821041 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00f6b0 con 0x7fe3d01008d0 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.280+0000 7fe3d77a7640 1 -- 172.21.15.31:0/2491821041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d01a7220 con 0x7fe3d01008d0 2024-09-16T00:48:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.282+0000 7fe3c67fc640 1 -- 172.21.15.31:0/2491821041 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe3cc0109d0 con 0x7fe3d01008d0 2024-09-16T00:48:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.282+0000 7fe3d9b80740 1 Processor -- start 2024-09-16T00:48:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.282+0000 7fe3d9b80740 1 -- start start 2024-09-16T00:48:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.282+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 0x561d7a514de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.282+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a515320 0x561d7a51f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a521fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561d7a479200 con 0x561d7a51fbe0 2024-09-16T00:48:46.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561d7a41fdc0 con 0x561d7a515320 2024-09-16T00:48:46.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561d7a4751c0 con 0x561d7a514a00 2024-09-16T00:48:46.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d4d42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a521fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d4d42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a521fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45180/0 (socket says 172.21.15.31:45180) 2024-09-16T00:48:46.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d4d42640 1 -- 172.21.15.31:0/1471587345 learned_addr learned my addr 172.21.15.31:0/1471587345 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:46.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d4d42640 1 -- 172.21.15.31:0/1471587345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 msgr2=0x561d7a514de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/1471587345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 0x561d7a514de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/1471587345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 0x561d7a514de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d4d42640 1 -- 172.21.15.31:0/1471587345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a515320 msgr2=0x561d7a51f6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:48:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/1471587345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a515320 0x561d7a51f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.283+0000 7fe3d4d42640 1 -- 172.21.15.31:0/1471587345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7a1ea620 con 0x561d7a51fbe0 2024-09-16T00:48:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.284+0000 7fe3c67fc640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe3bc0a3240 0x7fe3bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.284+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/1471587345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a521fd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000bcb0 tx=0x7fe3c000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9d02eb11e376c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.284+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe3bc0a3240 0x7fe3bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.284+0000 7fe3c67fc640 1 -- 172.21.15.31:0/2491821041 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fe3cc0cfa00 con 0x7fe3d01008d0 2024-09-16T00:48:46.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.284+0000 7fe3bb7fe640 1 -- 172.21.15.31:0/1471587345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0015070 con 0x561d7a51fbe0 2024-09-16T00:48:46.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.284+0000 7fe3bb7fe640 1 -- 172.21.15.31:0/1471587345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c0005500 con 0x561d7a51fbe0 2024-09-16T00:48:46.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.284+0000 7fe3bb7fe640 1 -- 172.21.15.31:0/1471587345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00116a0 con 0x561d7a51fbe0 2024-09-16T00:48:46.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1471587345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 msgr2=0x561d7a521fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:46.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1471587345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a521fd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000bcb0 tx=0x7fe3c000f730 comp rx=0 tx=0).stop 2024-09-16T00:48:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe3bc0a3240 0x7fe3bc0a5700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006010 tx=0x7fe3c8005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1471587345 shutdown_connections 2024-09-16T00:48:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1471587345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a521fd0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1471587345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a515320 0x561d7a51f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1471587345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 0x561d7a514de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1471587345 >> 172.21.15.31:0/1471587345 conn(0x561d7a510f90 msgr2=0x561d7a511370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1471587345 shutdown_connections 2024-09-16T00:48:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1471587345 wait complete. 2024-09-16T00:48:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 -- start start 2024-09-16T00:48:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.285+0000 7fe3d9b80740 1 Processor -- start 2024-09-16T00:48:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 0x561d7a51f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a515320 0x561d7a51c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a51cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561d7a5227b0 con 0x561d7a51fbe0 2024-09-16T00:48:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561d7a413c50 con 0x561d7a515320 2024-09-16T00:48:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561d7a522930 con 0x561d7a514a00 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d9b80740 10 client.0 _ll_register_callbacks cb 0x561d7a425ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5d44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a51cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5d44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a51cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45196/0 (socket says 172.21.15.31:45196) 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5d44640 1 -- 172.21.15.31:0/1433705756 learned_addr learned my addr 172.21.15.31:0/1433705756 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a515320 0x561d7a51c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5543640 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 0x561d7a51f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5d44640 1 -- 172.21.15.31:0/1433705756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 msgr2=0x561d7a51f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 0x561d7a51f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5d44640 1 -- 172.21.15.31:0/1433705756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a515320 msgr2=0x561d7a51c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a515320 0x561d7a51c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5d44640 1 -- 172.21.15.31:0/1433705756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7a51d540 con 0x561d7a51fbe0 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.286+0000 7fe3d5543640 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 0x561d7a51f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a51cd10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe3c802f540 tx=0x7fe3c80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e553072046a92c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:48:46.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1433705756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8004590 con 0x561d7a51fbe0 2024-09-16T00:48:46.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1433705756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c0005500 con 0x561d7a51fbe0 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1433705756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fe3c8004730 con 0x561d7a51fbe0 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1433705756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe3c802f610 con 0x561d7a51fbe0 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d9b80740 10 client.16320 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1433705756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561d7a52e330 con 0x561d7a51fbe0 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1433705756 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c0005500 con 0x561d7a51fbe0 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1433705756 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fe3c8043030 con 0x561d7a51fbe0 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d9b80740 10 client.16320 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d9b80740 10 client.16320 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d9b80740 20 client.16320 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561d7a52f740 0x561d7a531b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d9b80740 4 client.16320 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:48:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.287+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1433705756 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d7a486770 con 0x561d7a52f740 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.288+0000 7fe3d5543640 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561d7a52f740 0x561d7a531b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.288+0000 7fe3d5543640 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561d7a52f740 0x561d7a531b90 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6030362879e200b server_cookie=fb4f39b32f715c4e in_seq=0 out_seq=0 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.288+0000 7fe3b8ff9640 10 client.16320 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1433705756 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d7a486770 con 0x561d7a52f740 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3b8ff9640 10 client.16320 handle_command_reply: tid=0 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 client.16320 shutdown 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1433705756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 msgr2=0x561d7a51cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a51cd10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe3c802f540 tx=0x7fe3c80095c0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1433705756 shutdown_connections 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a51cd10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a515320 0x561d7a51c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561d7a52f740 0x561d7a531b90 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1433705756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a514a00 0x561d7a51f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3b8ff9640 0 client.16320 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1433705756 >> 172.21.15.31:0/1433705756 conn(0x561d7a510f90 msgr2=0x561d7a5101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1433705756 shutdown_connections 2024-09-16T00:48:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1433705756 wait complete. 2024-09-16T00:48:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.289+0000 7fe3d9b80740 20 client.16320 trim_cache size 0 max 16384 2024-09-16T00:48:46.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.306+0000 7fe3d9b80740 1 Processor -- start 2024-09-16T00:48:46.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.306+0000 7fe3d9b80740 1 -- start start 2024-09-16T00:48:46.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.306+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 0x561d7a51e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.306+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a515320 0x561d7a537460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.306+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a51fbe0 0x561d7a5379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01aef10 con 0x561d7a515320 2024-09-16T00:48:46.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01b1790 con 0x561d7a514a00 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01af580 con 0x561d7a51fbe0 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d4d42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a515320 0x561d7a537460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d4d42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a515320 0x561d7a537460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45212/0 (socket says 172.21.15.31:45212) 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d4d42640 1 -- 172.21.15.31:0/2446414396 learned_addr learned my addr 172.21.15.31:0/2446414396 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a51fbe0 0x561d7a5379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d5543640 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 0x561d7a51e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d4d42640 1 -- 172.21.15.31:0/2446414396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a51fbe0 msgr2=0x561d7a5379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a51fbe0 0x561d7a5379a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d4d42640 1 -- 172.21.15.31:0/2446414396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 msgr2=0x561d7a51e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 0x561d7a51e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d4d42640 1 -- 172.21.15.31:0/2446414396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7a532210 con 0x561d7a515320 2024-09-16T00:48:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a51fbe0 0x561d7a5379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d5543640 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 0x561d7a51e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a515320 0x561d7a537460 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0004cd0 tx=0x7fe3c0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee57e9afe22aa65 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3baffd640 1 -- 172.21.15.31:0/2446414396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0029070 con 0x561d7a515320 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3baffd640 1 -- 172.21.15.31:0/2446414396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c0004880 con 0x561d7a515320 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.307+0000 7fe3baffd640 1 -- 172.21.15.31:0/2446414396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0015050 con 0x561d7a515320 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 -- 172.21.15.31:0/2446414396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a515320 msgr2=0x561d7a537460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a515320 0x561d7a537460 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0004cd0 tx=0x7fe3c0004d00 comp rx=0 tx=0).stop 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 -- 172.21.15.31:0/2446414396 shutdown_connections 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a51fbe0 0x561d7a5379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a515320 0x561d7a537460 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/2446414396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 0x561d7a51e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 -- 172.21.15.31:0/2446414396 >> 172.21.15.31:0/2446414396 conn(0x561d7a510f90 msgr2=0x561d7a51e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:46.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 -- 172.21.15.31:0/2446414396 shutdown_connections 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 -- 172.21.15.31:0/2446414396 wait complete. 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 -- start start 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.308+0000 7fe3d9b80740 1 Processor -- start 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 0x561d7a5259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a515320 0x561d7a525f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d9b80740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a526460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01afc50 con 0x561d7a51fbe0 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01b10c0 con 0x561d7a514a00 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d9b80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01b0320 con 0x561d7a515320 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d9b80740 10 client.0 _ll_register_callbacks cb 0x7fe3c8003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d5d44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a526460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d5d44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a526460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45218/0 (socket says 172.21.15.31:45218) 2024-09-16T00:48:46.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d5d44640 1 -- 172.21.15.31:0/1241391101 learned_addr learned my addr 172.21.15.31:0/1241391101 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d4d42640 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a515320 0x561d7a525f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d5543640 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 0x561d7a5259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d5d44640 1 -- 172.21.15.31:0/1241391101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a515320 msgr2=0x561d7a525f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a515320 0x561d7a525f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d5d44640 1 -- 172.21.15.31:0/1241391101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 msgr2=0x561d7a5259e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 0x561d7a5259e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.309+0000 7fe3d5d44640 1 -- 172.21.15.31:0/1241391101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7a599540 con 0x561d7a51fbe0 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d5d44640 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a526460 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8031040 tx=0x7fe3c8030f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940af5b3e6e8856e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1241391101 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8004610 con 0x561d7a51fbe0 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1241391101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c0004880 con 0x561d7a51fbe0 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1241391101 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fe3c802f7b0 con 0x561d7a51fbe0 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1241391101 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe3c80307d0 con 0x561d7a51fbe0 2024-09-16T00:48:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d9b80740 10 client.16326 fetch_fsmap learned FSMap version 20 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1241391101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561d7a59a750 con 0x561d7a51fbe0 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1241391101 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c0004880 con 0x561d7a51fbe0 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1241391101 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fe3c8043070 con 0x561d7a51fbe0 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d9b80740 10 client.16326 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d9b80740 10 client.16326 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d9b80740 20 client.16326 populate_metadata read hostname 'smithi031' 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561d7a59ae70 0x561d7a59d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d9b80740 4 client.16326 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.310+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1241391101 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x561d7a480c80 con 0x561d7a59ae70 2024-09-16T00:48:46.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.311+0000 7fe3d5543640 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561d7a59ae70 0x561d7a59d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:46.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.311+0000 7fe3b8ff9640 10 client.16326 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.311+0000 7fe3d5543640 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561d7a59ae70 0x561d7a59d260 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a49a2d2679b7706 server_cookie=b6cb423659a5cff2 in_seq=0 out_seq=0 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.311+0000 7fe3b8ff9640 1 -- 172.21.15.31:0/1241391101 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x561d7a480c80 con 0x561d7a59ae70 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.311+0000 7fe3b8ff9640 10 client.16326 handle_command_reply: tid=0 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.311+0000 7fe3d9b80740 1 client.16326 shutdown 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.311+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1241391101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 msgr2=0x561d7a526460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.311+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a526460 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8031040 tx=0x7fe3c8030f20 comp rx=0 tx=0).stop 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1241391101 shutdown_connections 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561d7a51fbe0 0x561d7a526460 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x561d7a59ae70 0x561d7a59d260 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d7a515320 0x561d7a525f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3b8ff9640 0 client.16326 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d9b80740 1 --2- 172.21.15.31:0/1241391101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d7a514a00 0x561d7a5259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1241391101 >> 172.21.15.31:0/1241391101 conn(0x561d7a510f90 msgr2=0x561d7a5101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:46.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1241391101 shutdown_connections 2024-09-16T00:48:46.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d9b80740 1 -- 172.21.15.31:0/1241391101 wait complete. 2024-09-16T00:48:46.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d9b80740 20 client.16326 trim_cache size 0 max 16384 2024-09-16T00:48:46.319 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:48:46.319 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 20, 2024-09-16T00:48:46.319 INFO:teuthology.orchestra.run.smithi031.stdout: "leader": 24482, 2024-09-16T00:48:46.319 INFO:teuthology.orchestra.run.smithi031.stdout: "set_version": 24, 2024-09-16T00:48:46.319 INFO:teuthology.orchestra.run.smithi031.stdout: "sets": {} 2024-09-16T00:48:46.319 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:48:46.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 -- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe3bc0a3240 msgr2=0x7fe3bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:46.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe3bc0a3240 0x7fe3bc0a5700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006010 tx=0x7fe3c8005ef0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 -- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d01008d0 msgr2=0x7fe3d019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d01008d0 0x7fe3d019a6a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00d950 tx=0x7fe3cc00de10 comp rx=0 tx=0).stop 2024-09-16T00:48:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 -- 172.21.15.31:0/2491821041 shutdown_connections 2024-09-16T00:48:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fe3bc0a3240 0x7fe3bc0a5700 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3d010e480 0x7fe3d019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3d0101280 0x7fe3d019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 --2- 172.21.15.31:0/2491821041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d01008d0 0x7fe3d019a6a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.312+0000 7fe3d77a7640 1 -- 172.21.15.31:0/2491821041 >> 172.21.15.31:0/2491821041 conn(0x7fe3d00fc7a0 msgr2=0x7fe3d00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.313+0000 7fe3d77a7640 1 -- 172.21.15.31:0/2491821041 shutdown_connections 2024-09-16T00:48:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:48:46.313+0000 7fe3d77a7640 1 -- 172.21.15.31:0/2491821041 wait complete. 2024-09-16T00:48:47.341 INFO:tasks.quiescer:done joining Quiescers 2024-09-16T00:48:47.342 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-16T00:48:47.353 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-16T00:48:47.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:47 smithi081 ceph-mon[35539]: pgmap v585: 129 pgs: 129 active+clean; 194 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 766 op/s 2024-09-16T00:48:47.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:47 smithi102 ceph-mon[35364]: pgmap v585: 129 pgs: 129 active+clean; 194 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 766 op/s 2024-09-16T00:48:47.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:47 smithi031 ceph-mon[30956]: pgmap v585: 129 pgs: 129 active+clean; 194 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 766 op/s 2024-09-16T00:48:49.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:49 smithi081 ceph-mon[35539]: pgmap v586: 129 pgs: 129 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 696 op/s 2024-09-16T00:48:49.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:49 smithi102 ceph-mon[35364]: pgmap v586: 129 pgs: 129 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 696 op/s 2024-09-16T00:48:49.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:49 smithi031 ceph-mon[30956]: pgmap v586: 129 pgs: 129 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 696 op/s 2024-09-16T00:48:51.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:51 smithi081 ceph-mon[35539]: pgmap v587: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 837 KiB/s wr, 517 op/s 2024-09-16T00:48:51.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:51 smithi102 ceph-mon[35364]: pgmap v587: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 837 KiB/s wr, 517 op/s 2024-09-16T00:48:51.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:51 smithi031 ceph-mon[30956]: pgmap v587: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 837 KiB/s wr, 517 op/s 2024-09-16T00:48:53.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:53 smithi081 ceph-mon[35539]: pgmap v588: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 813 KiB/s wr, 380 op/s 2024-09-16T00:48:53.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:53 smithi102 ceph-mon[35364]: pgmap v588: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 813 KiB/s wr, 380 op/s 2024-09-16T00:48:53.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:53 smithi031 ceph-mon[30956]: pgmap v588: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 813 KiB/s wr, 380 op/s 2024-09-16T00:48:55.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:55 smithi081 ceph-mon[35539]: pgmap v589: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 481 KiB/s wr, 276 op/s 2024-09-16T00:48:55.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:55 smithi102 ceph-mon[35364]: pgmap v589: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 481 KiB/s wr, 276 op/s 2024-09-16T00:48:55.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:55 smithi031 ceph-mon[30956]: pgmap v589: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 481 KiB/s wr, 276 op/s 2024-09-16T00:48:57.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:57 smithi081 ceph-mon[35539]: pgmap v590: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 KiB/s wr, 88 op/s 2024-09-16T00:48:57.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:57 smithi102 ceph-mon[35364]: pgmap v590: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 KiB/s wr, 88 op/s 2024-09-16T00:48:57.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:57 smithi031 ceph-mon[30956]: pgmap v590: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 KiB/s wr, 88 op/s 2024-09-16T00:48:59.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:48:59 smithi081 ceph-mon[35539]: pgmap v591: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 50 op/s 2024-09-16T00:48:59.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:48:59 smithi102 ceph-mon[35364]: pgmap v591: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 50 op/s 2024-09-16T00:48:59.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:48:59 smithi031 ceph-mon[30956]: pgmap v591: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 50 op/s 2024-09-16T00:49:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:01 smithi081 ceph-mon[35539]: pgmap v592: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-16T00:49:01.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:01 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:49:01.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:01 smithi102 ceph-mon[35364]: pgmap v592: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-16T00:49:01.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:01 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:49:01.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:01 smithi031 ceph-mon[30956]: pgmap v592: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-16T00:49:01.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:01 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:49:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:49:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:49:02.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:02 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:49:02.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:49:02.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:49:02.550 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:02 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:49:02.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:49:02.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:49:02.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:02 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' 2024-09-16T00:49:02.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:49:03.242 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 -- 172.21.15.31:0/1288524928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c106f00 msgr2=0x7fbf3c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 --2- 172.21.15.31:0/1288524928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c106f00 0x7fbf3c10ea80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009ca0 tx=0x7fbf3402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 -- 172.21.15.31:0/1288524928 shutdown_connections 2024-09-16T00:49:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 --2- 172.21.15.31:0/1288524928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c106f00 0x7fbf3c10ea80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 --2- 172.21.15.31:0/1288524928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf3c102780 0x7fbf3c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 --2- 172.21.15.31:0/1288524928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c101dd0 0x7fbf3c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 -- 172.21.15.31:0/1288524928 >> 172.21.15.31:0/1288524928 conn(0x7fbf3c0fd8b0 msgr2=0x7fbf3c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 -- 172.21.15.31:0/1288524928 shutdown_connections 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 -- 172.21.15.31:0/1288524928 wait complete. 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 Processor -- start 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.754+0000 7fbf4291d640 1 -- start start 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf4291d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf3c101dd0 0x7fbf3c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf4291d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c102780 0x7fbf3c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf4291d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c106f00 0x7fbf3c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf4291d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c111f60 con 0x7fbf3c102780 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf4291d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c111de0 con 0x7fbf3c106f00 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf4291d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1120e0 con 0x7fbf3c101dd0 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c102780 0x7fbf3c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c102780 0x7fbf3c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34144/0 (socket says 172.21.15.31:34144) 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/133001701 learned_addr learned my addr 172.21.15.31:0/133001701 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf3c101dd0 0x7fbf3c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf40eba640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c106f00 0x7fbf3c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/133001701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf3c101dd0 msgr2=0x7fbf3c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf3c101dd0 0x7fbf3c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/133001701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c106f00 msgr2=0x7fbf3c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c106f00 0x7fbf3c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/133001701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c1a6870 con 0x7fbf3c102780 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.755+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c102780 0x7fbf3c074a30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d770 tx=0x7fbf3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.756+0000 7fbf397fa640 1 -- 172.21.15.31:0/133001701 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30017070 con 0x7fbf3c102780 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.756+0000 7fbf4291d640 1 -- 172.21.15.31:0/133001701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c1a6b60 con 0x7fbf3c102780 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.756+0000 7fbf397fa640 1 -- 172.21.15.31:0/133001701 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf30005500 con 0x7fbf3c102780 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.756+0000 7fbf397fa640 1 -- 172.21.15.31:0/133001701 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30004e00 con 0x7fbf3c102780 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.756+0000 7fbf4291d640 1 -- 172.21.15.31:0/133001701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c1a70a0 con 0x7fbf3c102780 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.758+0000 7fbf397fa640 1 -- 172.21.15.31:0/133001701 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbf30004fa0 con 0x7fbf3c102780 2024-09-16T00:49:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.758+0000 7fbf44cf6740 1 Processor -- start 2024-09-16T00:49:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.758+0000 7fbf44cf6740 1 -- start start 2024-09-16T00:49:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.758+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405506880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.758+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405506dc0 0x55b405511140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.758+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 0x55b405513a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.758+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b40546c2e0 con 0x55b4055064a0 2024-09-16T00:49:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.758+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b405413ce0 con 0x55b405506dc0 2024-09-16T00:49:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.758+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b4054682a0 con 0x55b405511680 2024-09-16T00:49:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405506880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405506880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34148/0 (socket says 172.21.15.31:34148) 2024-09-16T00:49:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 -- 172.21.15.31:0/3810851359 learned_addr learned my addr 172.21.15.31:0/3810851359 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/3810851359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405506dc0 0x55b405511140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf397fa640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf240a3240 0x7fbf240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/3810851359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 0x55b405513a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 -- 172.21.15.31:0/3810851359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 msgr2=0x55b405513a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 --2- 172.21.15.31:0/3810851359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 0x55b405513a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 -- 172.21.15.31:0/3810851359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405506dc0 msgr2=0x55b405511140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 --2- 172.21.15.31:0/3810851359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405506dc0 0x55b405511140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 -- 172.21.15.31:0/3810851359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4053f8440 con 0x55b4055064a0 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf397fa640 1 -- 172.21.15.31:0/133001701 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fbf300cf6e0 con 0x7fbf3c102780 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf240a3240 0x7fbf240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf40eba640 1 --2- 172.21.15.31:0/3810851359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405506880 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbf34005cc0 tx=0x7fbf34009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bb232b4b18277d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.759+0000 7fbf227fc640 1 -- 172.21.15.31:0/3810851359 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34041040 con 0x55b4055064a0 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf227fc640 1 -- 172.21.15.31:0/3810851359 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf34036070 con 0x55b4055064a0 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf227fc640 1 -- 172.21.15.31:0/3810851359 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3402f8c0 con 0x55b4055064a0 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf40eba640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf240a3240 0x7fbf240a5700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbf3404c000 tx=0x7fbf3404d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:03.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3810851359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 msgr2=0x55b405506880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3810851359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405506880 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbf34005cc0 tx=0x7fbf34009880 comp rx=0 tx=0).stop 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3810851359 shutdown_connections 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3810851359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 0x55b405513a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3810851359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405506dc0 0x55b405511140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3810851359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405506880 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3810851359 >> 172.21.15.31:0/3810851359 conn(0x55b405502a30 msgr2=0x55b405502e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3810851359 shutdown_connections 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.760+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3810851359 wait complete. 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf44cf6740 1 -- start start 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf44cf6740 1 Processor -- start 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405510fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405506dc0 0x55b40550e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40550e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b405514250 con 0x55b4055064a0 2024-09-16T00:49:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b405427050 con 0x55b405511680 2024-09-16T00:49:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b4055143d0 con 0x55b405506dc0 2024-09-16T00:49:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf44cf6740 10 client.0 _ll_register_callbacks cb 0x55b405467160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:49:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405510fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405510fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34164/0 (socket says 172.21.15.31:34164) 2024-09-16T00:49:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf3bfff640 1 -- 172.21.15.31:0/3099601862 learned_addr learned my addr 172.21.15.31:0/3099601862 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:03.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405506dc0 0x55b40550e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.761+0000 7fbf40eba640 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40550e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf3bfff640 1 -- 172.21.15.31:0/3099601862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405506dc0 msgr2=0x55b40550e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405506dc0 0x55b40550e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf3bfff640 1 -- 172.21.15.31:0/3099601862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 msgr2=0x55b40550e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40550e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf3bfff640 1 -- 172.21.15.31:0/3099601862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4051dd3c0 con 0x55b4055064a0 2024-09-16T00:49:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf40eba640 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40550e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:03.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405510fa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00bad0 tx=0x7fbf2c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba7783d30f0984c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:49:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/3099601862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c005020 con 0x55b4055064a0 2024-09-16T00:49:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/3099601862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fbf2c0051c0 con 0x55b4055064a0 2024-09-16T00:49:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3099601862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf34036070 con 0x55b4055064a0 2024-09-16T00:49:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/3099601862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbf2c005400 con 0x55b4055064a0 2024-09-16T00:49:03.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf44cf6740 10 client.16338 fetch_fsmap learned FSMap version 20 2024-09-16T00:49:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3099601862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b40551fcb0 con 0x55b4055064a0 2024-09-16T00:49:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/3099601862 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf34036070 con 0x55b4055064a0 2024-09-16T00:49:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/3099601862 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fbf2c01b6d0 con 0x55b4055064a0 2024-09-16T00:49:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf44cf6740 10 client.16338 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:49:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf44cf6740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:49:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf44cf6740 20 client.16338 populate_metadata read hostname 'smithi031' 2024-09-16T00:49:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.762+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55b405521170 0x55b4055235a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.763+0000 7fbf44cf6740 4 client.16338 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:49:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.763+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3099601862 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b405479850 con 0x55b405521170 2024-09-16T00:49:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.763+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55b405521170 0x55b4055235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.763+0000 7fbf1b7fe640 10 client.16338 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:49:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.763+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55b405521170 0x55b4055235a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08fab5e0e3c6c10 server_cookie=f824ee3ca099e54e in_seq=0 out_seq=0 2024-09-16T00:49:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/3099601862 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b405479850 con 0x55b405521170 2024-09-16T00:49:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf1b7fe640 10 client.16338 handle_command_reply: tid=0 2024-09-16T00:49:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 client.16338 shutdown 2024-09-16T00:49:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3099601862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 msgr2=0x55b405510fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405510fa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00bad0 tx=0x7fbf2c00bed0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3099601862 shutdown_connections 2024-09-16T00:49:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40550e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55b405521170 0x55b4055235a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405506dc0 0x55b40550e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/3099601862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b4055064a0 0x55b405510fa0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf1b7fe640 0 client.16338 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:49:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3099601862 >> 172.21.15.31:0/3099601862 conn(0x55b405502a30 msgr2=0x55b405501b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3099601862 shutdown_connections 2024-09-16T00:49:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.764+0000 7fbf44cf6740 1 -- 172.21.15.31:0/3099601862 wait complete. 2024-09-16T00:49:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.765+0000 7fbf44cf6740 20 client.16338 trim_cache size 0 max 16384 2024-09-16T00:49:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf44cf6740 1 Processor -- start 2024-09-16T00:49:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf44cf6740 1 -- start start 2024-09-16T00:49:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4055064a0 0x55b40551d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405528e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 0x55b4055293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1aee20 con 0x55b405506dc0 2024-09-16T00:49:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1b16a0 con 0x55b4055064a0 2024-09-16T00:49:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1af490 con 0x55b405511680 2024-09-16T00:49:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405528e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405528e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34176/0 (socket says 172.21.15.31:34176) 2024-09-16T00:49:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/743070053 learned_addr learned my addr 172.21.15.31:0/743070053 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4055064a0 0x55b40551d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/743070053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 msgr2=0x55b4055293b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf40eba640 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 0x55b4055293b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 0x55b4055293b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/743070053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4055064a0 msgr2=0x55b40551d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4055064a0 0x55b40551d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/743070053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b405523c20 con 0x55b405506dc0 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4055064a0 0x55b40551d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.782+0000 7fbf40eba640 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 0x55b4055293b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405528e70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbf30097d50 tx=0x7fbf300ccca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6331f01fba513599 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf21ffb640 1 -- 172.21.15.31:0/743070053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30020040 con 0x55b405506dc0 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf21ffb640 1 -- 172.21.15.31:0/743070053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf300a4560 con 0x55b405506dc0 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf21ffb640 1 -- 172.21.15.31:0/743070053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30009cc0 con 0x55b405506dc0 2024-09-16T00:49:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf44cf6740 1 -- 172.21.15.31:0/743070053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 msgr2=0x55b405528e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405528e70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbf30097d50 tx=0x7fbf300ccca0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf44cf6740 1 -- 172.21.15.31:0/743070053 shutdown_connections 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b405511680 0x55b4055293b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405528e70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.783+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/743070053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4055064a0 0x55b40551d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 -- 172.21.15.31:0/743070053 >> 172.21.15.31:0/743070053 conn(0x55b405502a30 msgr2=0x55b40550cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 -- 172.21.15.31:0/743070053 shutdown_connections 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 -- 172.21.15.31:0/743070053 wait complete. 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 -- start start 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 Processor -- start 2024-09-16T00:49:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4055064a0 0x55b4055157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405515d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40558ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1afb60 con 0x55b405506dc0 2024-09-16T00:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1b0fd0 con 0x55b405511680 2024-09-16T00:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1b0230 con 0x55b4055064a0 2024-09-16T00:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.784+0000 7fbf44cf6740 10 client.0 _ll_register_callbacks cb 0x7fbf2c0040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405515d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405515d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34186/0 (socket says 172.21.15.31:34186) 2024-09-16T00:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/2121018554 learned_addr learned my addr 172.21.15.31:0/2121018554 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4055064a0 0x55b4055157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf40eba640 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40558ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/2121018554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4055064a0 msgr2=0x55b4055157d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4055064a0 0x55b4055157d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/2121018554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 msgr2=0x55b40558ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40558ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3b7fe640 1 -- 172.21.15.31:0/2121018554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40558b380 con 0x55b405506dc0 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4055064a0 0x55b4055157d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf40eba640 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40558ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf3b7fe640 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405515d10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbf30021000 tx=0x7fbf3009c160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3404caf97d411147 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/2121018554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf300b9060 con 0x55b405506dc0 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/2121018554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fbf300ccee0 con 0x55b405506dc0 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf44cf6740 1 -- 172.21.15.31:0/2121018554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf300a4560 con 0x55b405506dc0 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/2121018554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbf300b3070 con 0x55b405506dc0 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf44cf6740 10 client.16344 fetch_fsmap learned FSMap version 20 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.785+0000 7fbf44cf6740 1 -- 172.21.15.31:0/2121018554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b40558c480 con 0x55b405506dc0 2024-09-16T00:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.786+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/2121018554 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf300a4560 con 0x55b405506dc0 2024-09-16T00:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.786+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/2121018554 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fbf30097f60 con 0x55b405506dc0 2024-09-16T00:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.786+0000 7fbf44cf6740 10 client.16344 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.786+0000 7fbf44cf6740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.786+0000 7fbf44cf6740 20 client.16344 populate_metadata read hostname 'smithi031' 2024-09-16T00:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.786+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55b40558cb30 0x55b40558ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.786+0000 7fbf44cf6740 4 client.16344 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.786+0000 7fbf44cf6740 1 -- 172.21.15.31:0/2121018554 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b40550eb20 con 0x55b40558cb30 2024-09-16T00:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.786+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55b40558cb30 0x55b40558ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf3bfff640 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55b40558cb30 0x55b40558ef50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bef5dd7d036bce33 server_cookie=e46beaedc06d2d97 in_seq=0 out_seq=0 2024-09-16T00:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf1b7fe640 10 client.16344 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf1b7fe640 1 -- 172.21.15.31:0/2121018554 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b40550eb20 con 0x55b40558cb30 2024-09-16T00:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf1b7fe640 10 client.16344 handle_command_reply: tid=0 2024-09-16T00:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 client.16344 shutdown 2024-09-16T00:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 -- 172.21.15.31:0/2121018554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 msgr2=0x55b405515d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405515d10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbf30021000 tx=0x7fbf3009c160 comp rx=0 tx=0).stop 2024-09-16T00:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 -- 172.21.15.31:0/2121018554 shutdown_connections 2024-09-16T00:49:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b405511680 0x55b40558ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b405506dc0 0x55b405515d10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55b40558cb30 0x55b40558ef50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 --2- 172.21.15.31:0/2121018554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4055064a0 0x55b4055157d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 -- 172.21.15.31:0/2121018554 >> 172.21.15.31:0/2121018554 conn(0x55b405502a30 msgr2=0x55b405507ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf1b7fe640 0 client.16344 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:49:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 -- 172.21.15.31:0/2121018554 shutdown_connections 2024-09-16T00:49:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 1 -- 172.21.15.31:0/2121018554 wait complete. 2024-09-16T00:49:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.787+0000 7fbf44cf6740 20 client.16344 trim_cache size 0 max 16384 2024-09-16T00:49:03.801 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:49:03.801 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-16T00:49:03.801 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2024-09-16T00:49:03.802 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:49:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 -- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf240a3240 msgr2=0x7fbf240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:03.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf240a3240 0x7fbf240a5700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbf3404c000 tx=0x7fbf3404d000 comp rx=0 tx=0).stop 2024-09-16T00:49:03.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 -- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c102780 msgr2=0x7fbf3c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:03.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c102780 0x7fbf3c074a30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d770 tx=0x7fbf3000dc30 comp rx=0 tx=0).stop 2024-09-16T00:49:03.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 -- 172.21.15.31:0/133001701 shutdown_connections 2024-09-16T00:49:03.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c106f00 0x7fbf3c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf3c102780 0x7fbf3c074a30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fbf240a3240 0x7fbf240a5700 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 --2- 172.21.15.31:0/133001701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf3c101dd0 0x7fbf3c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 -- 172.21.15.31:0/133001701 >> 172.21.15.31:0/133001701 conn(0x7fbf3c0fd8b0 msgr2=0x7fbf3c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 -- 172.21.15.31:0/133001701 shutdown_connections 2024-09-16T00:49:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:03.788+0000 7fbf4291d640 1 -- 172.21.15.31:0/133001701 wait complete. 2024-09-16T00:49:03.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:03 smithi081 ceph-mon[35539]: pgmap v593: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-16T00:49:04.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:03 smithi102 ceph-mon[35364]: pgmap v593: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-16T00:49:04.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:03 smithi031 ceph-mon[30956]: pgmap v593: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-16T00:49:04.702 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:49:04.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:49:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:05.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.487+0000 7f7e88ca1640 1 -- 172.21.15.31:0/3216642308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 msgr2=0x7f7e8410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:05.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.487+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/3216642308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 0x7f7e8410bb50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009900 tx=0x7f7e7802a480 comp rx=0 tx=0).stop 2024-09-16T00:49:05.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.487+0000 7f7e88ca1640 1 -- 172.21.15.31:0/3216642308 shutdown_connections 2024-09-16T00:49:05.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.487+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/3216642308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e8410c170 0x7f7e8410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.487+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/3216642308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 0x7f7e8410bb50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.487+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/3216642308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e84102d40 0x7f7e84103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.487+0000 7f7e88ca1640 1 -- 172.21.15.31:0/3216642308 >> 172.21.15.31:0/3216642308 conn(0x7f7e840fc7a0 msgr2=0x7f7e840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.487+0000 7f7e88ca1640 1 -- 172.21.15.31:0/3216642308 shutdown_connections 2024-09-16T00:49:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.487+0000 7f7e88ca1640 1 -- 172.21.15.31:0/3216642308 wait complete. 2024-09-16T00:49:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e88ca1640 1 Processor -- start 2024-09-16T00:49:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e88ca1640 1 -- start start 2024-09-16T00:49:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e88ca1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e84102d40 0x7f7e8419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e88ca1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 0x7f7e8419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e88ca1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e8410c170 0x7f7e8419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e88ca1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84111fb0 con 0x7f7e84103660 2024-09-16T00:49:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e88ca1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84111e30 con 0x7f7e84102d40 2024-09-16T00:49:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e88ca1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84112130 con 0x7f7e8410c170 2024-09-16T00:49:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e81d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 0x7f7e8419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e81d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 0x7f7e8419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34208/0 (socket says 172.21.15.31:34208) 2024-09-16T00:49:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e81d9b640 1 -- 172.21.15.31:0/452811454 learned_addr learned my addr 172.21.15.31:0/452811454 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.488+0000 7f7e8259c640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e84102d40 0x7f7e8419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:05.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e82d9d640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e8410c170 0x7f7e8419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:05.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e81d9b640 1 -- 172.21.15.31:0/452811454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e8410c170 msgr2=0x7f7e8419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:05.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e81d9b640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e8410c170 0x7f7e8419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e81d9b640 1 -- 172.21.15.31:0/452811454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e84102d40 msgr2=0x7f7e8419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:05.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e81d9b640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e84102d40 0x7f7e8419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e81d9b640 1 -- 172.21.15.31:0/452811454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8419f600 con 0x7f7e84103660 2024-09-16T00:49:05.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e81d9b640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 0x7f7e8419ab90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005ab0 tx=0x7f7e780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:05.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e737fe640 1 -- 172.21.15.31:0/452811454 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7803f030 con 0x7f7e84103660 2024-09-16T00:49:05.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e737fe640 1 -- 172.21.15.31:0/452811454 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e78003ce0 con 0x7f7e84103660 2024-09-16T00:49:05.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8419f890 con 0x7f7e84103660 2024-09-16T00:49:05.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e737fe640 1 -- 172.21.15.31:0/452811454 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78035070 con 0x7f7e84103660 2024-09-16T00:49:05.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.489+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e841a71d0 con 0x7f7e84103660 2024-09-16T00:49:05.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.490+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e84103120 con 0x7f7e84103660 2024-09-16T00:49:05.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.491+0000 7f7e737fe640 1 -- 172.21.15.31:0/452811454 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7e78042030 con 0x7f7e84103660 2024-09-16T00:49:05.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.492+0000 7f7e737fe640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7e6c0a32c0 0x7f7e6c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:05.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.492+0000 7f7e8259c640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7e6c0a32c0 0x7f7e6c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:05.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.493+0000 7f7e737fe640 1 -- 172.21.15.31:0/452811454 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f7e780f0720 con 0x7f7e84103660 2024-09-16T00:49:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.493+0000 7f7e8259c640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7e6c0a32c0 0x7f7e6c0a5780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400abe0 tx=0x7f7e7400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:05.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.496+0000 7f7e737fe640 1 -- 172.21.15.31:0/452811454 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e780bd030 con 0x7f7e84103660 2024-09-16T00:49:05.662 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:05 smithi031 ceph-mon[30956]: pgmap v594: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-09-16T00:49:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.716+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e8419b9b0 con 0x7f7e84103660 2024-09-16T00:49:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.717+0000 7f7e737fe640 1 -- 172.21.15.31:0/452811454 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f7e8419b9b0 con 0x7f7e84103660 2024-09-16T00:49:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:49:05.720 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:49:05.720 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:49:05.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.719+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7e6c0a32c0 msgr2=0x7f7e6c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.719+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7e6c0a32c0 0x7f7e6c0a5780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400abe0 tx=0x7f7e7400a2d0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.719+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 msgr2=0x7f7e8419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.719+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 0x7f7e8419ab90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005ab0 tx=0x7f7e780042f0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.720+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 shutdown_connections 2024-09-16T00:49:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.720+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7e6c0a32c0 0x7f7e6c0a5780 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.720+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e8410c170 0x7f7e8419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.720+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e84103660 0x7f7e8419ab90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.720+0000 7f7e88ca1640 1 --2- 172.21.15.31:0/452811454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e84102d40 0x7f7e8419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.720+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 >> 172.21.15.31:0/452811454 conn(0x7f7e840fc7a0 msgr2=0x7f7e8410d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.720+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 shutdown_connections 2024-09-16T00:49:05.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:05.720+0000 7f7e88ca1640 1 -- 172.21.15.31:0/452811454 wait complete. 2024-09-16T00:49:05.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:05 smithi081 ceph-mon[35539]: pgmap v594: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-09-16T00:49:06.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:05 smithi102 ceph-mon[35364]: pgmap v594: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-09-16T00:49:06.494 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:49:06.755 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:06.876 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:06 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/452811454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:49:06.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:06 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/452811454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:49:07.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:06 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/452811454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:49:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.283+0000 7faa9166a640 1 -- 172.21.15.31:0/2520313430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c101de0 msgr2=0x7faa8c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.283+0000 7faa9166a640 1 --2- 172.21.15.31:0/2520313430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c101de0 0x7faa8c1021c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009900 tx=0x7faa7c02a410 comp rx=0 tx=0).stop 2024-09-16T00:49:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 -- 172.21.15.31:0/2520313430 shutdown_connections 2024-09-16T00:49:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 --2- 172.21.15.31:0/2520313430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa8c106f10 0x7faa8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 --2- 172.21.15.31:0/2520313430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa8c102790 0x7faa8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 --2- 172.21.15.31:0/2520313430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c101de0 0x7faa8c1021c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 -- 172.21.15.31:0/2520313430 >> 172.21.15.31:0/2520313430 conn(0x7faa8c0fd8c0 msgr2=0x7faa8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 -- 172.21.15.31:0/2520313430 shutdown_connections 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 -- 172.21.15.31:0/2520313430 wait complete. 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 Processor -- start 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 -- start start 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa8c101de0 0x7faa8c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c102790 0x7faa8c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa8c106f10 0x7faa8c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c111f70 con 0x7faa8c102790 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c111df0 con 0x7faa8c101de0 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.284+0000 7faa9166a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c1120f0 con 0x7faa8c106f10 2024-09-16T00:49:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c102790 0x7faa8c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c102790 0x7faa8c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34224/0 (socket says 172.21.15.31:34224) 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa83fff640 1 -- 172.21.15.31:0/1613975924 learned_addr learned my addr 172.21.15.31:0/1613975924 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa8affd640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa8c101de0 0x7faa8c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa8c106f10 0x7faa8c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa83fff640 1 -- 172.21.15.31:0/1613975924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa8c106f10 msgr2=0x7faa8c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa83fff640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa8c106f10 0x7faa8c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa83fff640 1 -- 172.21.15.31:0/1613975924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa8c101de0 msgr2=0x7faa8c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa83fff640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa8c101de0 0x7faa8c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa83fff640 1 -- 172.21.15.31:0/1613975924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c1a6990 con 0x7faa8c102790 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa8c106f10 0x7faa8c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa83fff640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c102790 0x7faa8c074a30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7faa7400e9f0 tx=0x7faa7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa88ff9640 1 -- 172.21.15.31:0/1613975924 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7400cd80 con 0x7faa8c102790 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa9166a640 1 -- 172.21.15.31:0/1613975924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8c1a6c80 con 0x7faa8c102790 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa88ff9640 1 -- 172.21.15.31:0/1613975924 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa74010910 con 0x7faa8c102790 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa88ff9640 1 -- 172.21.15.31:0/1613975924 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa740188f0 con 0x7faa8c102790 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.285+0000 7faa9166a640 1 -- 172.21.15.31:0/1613975924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8c1a71c0 con 0x7faa8c102790 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.288+0000 7faa88ff9640 1 -- 172.21.15.31:0/1613975924 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7faa74010ac0 con 0x7faa8c102790 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.288+0000 7faa93a43740 1 Processor -- start 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.288+0000 7faa93a43740 1 -- start start 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa93a43740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bacf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa93a43740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bacfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa93a43740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 0x55950badc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa93a43740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55950ba352f0 con 0x55950bacf5d0 2024-09-16T00:49:07.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa93a43740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55950b9dccf0 con 0x55950bada370 2024-09-16T00:49:07.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa93a43740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55950ba312b0 con 0x55950bad0b40 2024-09-16T00:49:07.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa88ff9640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faa780a3240 0x7faa780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bacfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bacfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34236/0 (socket says 172.21.15.31:34236) 2024-09-16T00:49:07.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8affd640 1 -- 172.21.15.31:0/536889377 learned_addr learned my addr 172.21.15.31:0/536889377 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:07.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa83fff640 1 --2- 172.21.15.31:0/536889377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 0x55950badc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa88ff9640 1 -- 172.21.15.31:0/1613975924 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7faa740d0120 con 0x7faa8c102790 2024-09-16T00:49:07.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/536889377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bacf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faa780a3240 0x7faa780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8affd640 1 -- 172.21.15.31:0/536889377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 msgr2=0x55950bacf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:07.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8affd640 1 --2- 172.21.15.31:0/536889377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bacf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8affd640 1 -- 172.21.15.31:0/536889377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 msgr2=0x55950badc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:07.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8affd640 1 --2- 172.21.15.31:0/536889377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 0x55950badc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8affd640 1 -- 172.21.15.31:0/536889377 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950b9c13a0 con 0x55950bacf5d0 2024-09-16T00:49:07.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8affd640 1 --2- 172.21.15.31:0/536889377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bacfa10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009900 tx=0x7faa7c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ec5595a8200907 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:07.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.289+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faa780a3240 0x7faa780a5700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa84009fd0 tx=0x7faa84009240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:07.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.290+0000 7faa817fa640 1 -- 172.21.15.31:0/536889377 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c040070 con 0x55950bacf5d0 2024-09-16T00:49:07.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.290+0000 7faa817fa640 1 -- 172.21.15.31:0/536889377 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa7c0318f0 con 0x55950bacf5d0 2024-09-16T00:49:07.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.290+0000 7faa817fa640 1 -- 172.21.15.31:0/536889377 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c035040 con 0x55950bacf5d0 2024-09-16T00:49:07.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.290+0000 7faa93a43740 1 -- 172.21.15.31:0/536889377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 msgr2=0x55950bacfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:07.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.290+0000 7faa93a43740 1 --2- 172.21.15.31:0/536889377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bacfa10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009900 tx=0x7faa7c005e00 comp rx=0 tx=0).stop 2024-09-16T00:49:07.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 -- 172.21.15.31:0/536889377 shutdown_connections 2024-09-16T00:49:07.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 --2- 172.21.15.31:0/536889377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 0x55950badc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 --2- 172.21.15.31:0/536889377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bacfa10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 --2- 172.21.15.31:0/536889377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bacf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 -- 172.21.15.31:0/536889377 >> 172.21.15.31:0/536889377 conn(0x55950ba98310 msgr2=0x55950bacbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:07.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 -- 172.21.15.31:0/536889377 shutdown_connections 2024-09-16T00:49:07.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 -- 172.21.15.31:0/536889377 wait complete. 2024-09-16T00:49:07.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 -- start start 2024-09-16T00:49:07.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 Processor -- start 2024-09-16T00:49:07.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 0x55950bad9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bad0b40 0x55950bad6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bada370 0x55950bad75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55950badd010 con 0x55950bad0b40 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55950b9f0060 con 0x55950bacf5d0 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55950badd190 con 0x55950bada370 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa93a43740 10 client.0 _ll_register_callbacks cb 0x55950b7438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bad0b40 0x55950bad6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bad0b40 0x55950bad6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54474/0 (socket says 172.21.15.31:54474) 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.291+0000 7faa83fff640 1 -- 172.21.15.31:0/863156157 learned_addr learned my addr 172.21.15.31:0/863156157 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa8affd640 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 0x55950bad9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bada370 0x55950bad75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa83fff640 1 -- 172.21.15.31:0/863156157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bada370 msgr2=0x55950bad75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa83fff640 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bada370 0x55950bad75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa83fff640 1 -- 172.21.15.31:0/863156157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 msgr2=0x55950bad9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa83fff640 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 0x55950bad9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa83fff640 1 -- 172.21.15.31:0/863156157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950b7a63c0 con 0x55950bad0b40 2024-09-16T00:49:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa8affd640 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 0x55950bad9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa83fff640 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bad0b40 0x55950bad6fe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faa7400cfd0 tx=0x7faa74014cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e8adb03a24432b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa727fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:49:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa727fc640 1 -- 172.21.15.31:0/863156157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7409c390 con 0x55950bad0b40 2024-09-16T00:49:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa727fc640 1 -- 172.21.15.31:0/863156157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7faa7409b7e0 con 0x55950bad0b40 2024-09-16T00:49:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa93a43740 1 -- 172.21.15.31:0/863156157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa7c0318f0 con 0x55950bad0b40 2024-09-16T00:49:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa727fc640 1 -- 172.21.15.31:0/863156157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7faa740a0740 con 0x55950bad0b40 2024-09-16T00:49:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa93a43740 10 client.16362 fetch_fsmap learned FSMap version 20 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa93a43740 1 -- 172.21.15.31:0/863156157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55950bae8ac0 con 0x55950bad0b40 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.292+0000 7faa727fc640 1 -- 172.21.15.31:0/863156157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa7c0318f0 con 0x55950bad0b40 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa727fc640 1 -- 172.21.15.31:0/863156157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7faa740980e0 con 0x55950bad0b40 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa93a43740 10 client.16362 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa93a43740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa93a43740 20 client.16362 populate_metadata read hostname 'smithi031' 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa93a43740 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55950bae9ed0 0x55950baec320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa93a43740 4 client.16362 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa93a43740 1 -- 172.21.15.31:0/863156157 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55950ba42860 con 0x55950bae9ed0 2024-09-16T00:49:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa8affd640 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55950bae9ed0 0x55950baec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa8affd640 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55950bae9ed0 0x55950baec320 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd338a7e90c7d4fa server_cookie=e1a322e9b4094d94 in_seq=0 out_seq=0 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.293+0000 7faa727fc640 10 client.16362 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa727fc640 1 -- 172.21.15.31:0/863156157 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55950ba42860 con 0x55950bae9ed0 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa727fc640 10 client.16362 handle_command_reply: tid=0 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 client.16362 shutdown 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 -- 172.21.15.31:0/863156157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bad0b40 msgr2=0x55950bad6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bad0b40 0x55950bad6fe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faa7400cfd0 tx=0x7faa74014cb0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 -- 172.21.15.31:0/863156157 shutdown_connections 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55950bae9ed0 0x55950baec320 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bada370 0x55950bad75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bad0b40 0x55950bad6fe0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa727fc640 0 client.16362 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:49:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 --2- 172.21.15.31:0/863156157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 0x55950bad9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 -- 172.21.15.31:0/863156157 >> 172.21.15.31:0/863156157 conn(0x55950ba98310 msgr2=0x55950bad1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 -- 172.21.15.31:0/863156157 shutdown_connections 2024-09-16T00:49:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 1 -- 172.21.15.31:0/863156157 wait complete. 2024-09-16T00:49:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.294+0000 7faa93a43740 20 client.16362 trim_cache size 0 max 16384 2024-09-16T00:49:07.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.307+0000 7faa93a43740 1 Processor -- start 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.307+0000 7faa93a43740 1 -- start start 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa93a43740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 0x55950bae62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa93a43740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bad68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa93a43740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bada370 0x55950bad2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa93a43740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c1aedf0 con 0x55950bada370 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa93a43740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c1b1670 con 0x55950bacf5d0 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa93a43740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c1af460 con 0x55950bad0b40 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bada370 0x55950bad2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bada370 0x55950bad2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54486/0 (socket says 172.21.15.31:54486) 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8b7fe640 1 -- 172.21.15.31:0/3443277525 learned_addr learned my addr 172.21.15.31:0/3443277525 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa83fff640 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bad68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8affd640 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 0x55950bae62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8b7fe640 1 -- 172.21.15.31:0/3443277525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 msgr2=0x55950bad68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bad68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8b7fe640 1 -- 172.21.15.31:0/3443277525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 msgr2=0x55950bae62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 0x55950bae62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8b7fe640 1 -- 172.21.15.31:0/3443277525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950bad7940 con 0x55950bada370 2024-09-16T00:49:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa83fff640 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bad68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.308+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bada370 0x55950bad2120 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faa84011fa0 tx=0x7faa84009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc6c8ee6a4b82fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa80ff9640 1 -- 172.21.15.31:0/3443277525 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8401fbc0 con 0x55950bada370 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa80ff9640 1 -- 172.21.15.31:0/3443277525 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa84003e50 con 0x55950bada370 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa80ff9640 1 -- 172.21.15.31:0/3443277525 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84004770 con 0x55950bada370 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa93a43740 1 -- 172.21.15.31:0/3443277525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bada370 msgr2=0x55950bad2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa93a43740 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bada370 0x55950bad2120 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faa84011fa0 tx=0x7faa84009710 comp rx=0 tx=0).stop 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa93a43740 1 -- 172.21.15.31:0/3443277525 shutdown_connections 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa93a43740 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bada370 0x55950bad2120 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa93a43740 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bad68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa93a43740 1 --2- 172.21.15.31:0/3443277525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bacf5d0 0x55950bae62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa93a43740 1 -- 172.21.15.31:0/3443277525 >> 172.21.15.31:0/3443277525 conn(0x55950ba98310 msgr2=0x55950bad60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa93a43740 1 -- 172.21.15.31:0/3443277525 shutdown_connections 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.309+0000 7faa93a43740 1 -- 172.21.15.31:0/3443277525 wait complete. 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa93a43740 1 -- start start 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa93a43740 1 Processor -- start 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa93a43740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bae26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa93a43740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bae2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa93a43740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 0x55950badda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa93a43740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c1afb30 con 0x55950bacf5d0 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa93a43740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c1b0fa0 con 0x55950bada370 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa93a43740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c1b0200 con 0x55950bad0b40 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa93a43740 10 client.0 _ll_register_callbacks cb 0x7faa74097f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa8affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bae26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa8affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bae26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54502/0 (socket says 172.21.15.31:54502) 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa8affd640 1 -- 172.21.15.31:0/2948243879 learned_addr learned my addr 172.21.15.31:0/2948243879 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.310+0000 7faa8b7fe640 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 0x55950badda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa83fff640 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bae2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa8affd640 1 -- 172.21.15.31:0/2948243879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 msgr2=0x55950bae2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa8affd640 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bae2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa8affd640 1 -- 172.21.15.31:0/2948243879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 msgr2=0x55950badda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa8affd640 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 0x55950badda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa8affd640 1 -- 172.21.15.31:0/2948243879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950bade2b0 con 0x55950bacf5d0 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa8affd640 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bae26f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55950baccac0 tx=0x7faa7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b72c6d50b1a29c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa727fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:49:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa727fc640 1 -- 172.21.15.31:0/2948243879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c040040 con 0x55950bacf5d0 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa727fc640 1 -- 172.21.15.31:0/2948243879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7faa7c0316f0 con 0x55950bacf5d0 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa93a43740 1 -- 172.21.15.31:0/2948243879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa84003e50 con 0x55950bacf5d0 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa727fc640 1 -- 172.21.15.31:0/2948243879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7faa7c060050 con 0x55950bacf5d0 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa93a43740 10 client.16368 fetch_fsmap learned FSMap version 20 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa93a43740 1 -- 172.21.15.31:0/2948243879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55950bb547a0 con 0x55950bacf5d0 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa727fc640 1 -- 172.21.15.31:0/2948243879 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa84003e50 con 0x55950bacf5d0 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.311+0000 7faa727fc640 1 -- 172.21.15.31:0/2948243879 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7faa7c04d040 con 0x55950bacf5d0 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 10 client.16368 fetch_fsmap finished waiting for FSMap version 20 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 20 client.16368 populate_metadata read hostname 'smithi031' 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55950bb54b00 0x55950bb56f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 4 client.16368 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:49:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 1 -- 172.21.15.31:0/2948243879 --> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55950ba3cd70 con 0x55950bb54b00 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa83fff640 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55950bb54b00 0x55950bb56f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa83fff640 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55950bb54b00 0x55950bb56f20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0fb9dbf2aac0d6 server_cookie=a21852b44d5cb77d in_seq=0 out_seq=0 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa727fc640 10 client.16368 ms_handle_connect on v2:172.21.15.81:6838/2243721734 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa727fc640 1 -- 172.21.15.31:0/2948243879 <== mds.0 v2:172.21.15.81:6838/2243721734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55950ba3cd70 con 0x55950bb54b00 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa727fc640 10 client.16368 handle_command_reply: tid=0 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 1 client.16368 shutdown 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 1 -- 172.21.15.31:0/2948243879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 msgr2=0x55950bae26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bae26f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55950baccac0 tx=0x7faa7c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 1 -- 172.21.15.31:0/2948243879 shutdown_connections 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55950bada370 0x55950badda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.312+0000 7faa93a43740 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.81:6838/2243721734,v1:172.21.15.81:6839/2243721734] conn(0x55950bb54b00 0x55950bb56f20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa93a43740 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950bad0b40 0x55950bae2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa93a43740 1 --2- 172.21.15.31:0/2948243879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55950bacf5d0 0x55950bae26f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa727fc640 0 client.16368 ms_handle_reset on v2:172.21.15.81:6838/2243721734 2024-09-16T00:49:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa93a43740 1 -- 172.21.15.31:0/2948243879 >> 172.21.15.31:0/2948243879 conn(0x55950ba98310 msgr2=0x55950bad8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa93a43740 1 -- 172.21.15.31:0/2948243879 shutdown_connections 2024-09-16T00:49:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa93a43740 1 -- 172.21.15.31:0/2948243879 wait complete. 2024-09-16T00:49:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa93a43740 20 client.16368 trim_cache size 0 max 16384 2024-09-16T00:49:07.329 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-16T00:49:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 -- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faa780a3240 msgr2=0x7faa780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faa780a3240 0x7faa780a5700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa84009fd0 tx=0x7faa84009240 comp rx=0 tx=0).stop 2024-09-16T00:49:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 -- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c102790 msgr2=0x7faa8c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c102790 0x7faa8c074a30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7faa7400e9f0 tx=0x7faa7400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 -- 172.21.15.31:0/1613975924 shutdown_connections 2024-09-16T00:49:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7faa780a3240 0x7faa780a5700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa8c106f10 0x7faa8c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa8c102790 0x7faa8c074a30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 --2- 172.21.15.31:0/1613975924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa8c101de0 0x7faa8c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 -- 172.21.15.31:0/1613975924 >> 172.21.15.31:0/1613975924 conn(0x7faa8c0fd8c0 msgr2=0x7faa8c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 -- 172.21.15.31:0/1613975924 shutdown_connections 2024-09-16T00:49:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:07.313+0000 7faa9166a640 1 -- 172.21.15.31:0/1613975924 wait complete. 2024-09-16T00:49:07.918 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:07 smithi031 ceph-mon[30956]: pgmap v595: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-16T00:49:07.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:07 smithi081 ceph-mon[35539]: pgmap v595: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-16T00:49:08.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:07 smithi102 ceph-mon[35364]: pgmap v595: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-16T00:49:08.099 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-16T00:49:08.100 INFO:tasks.fwd_scrub:done joining 2024-09-16T00:49:08.101 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-16T00:49:08.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:49:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.226+0000 7f077bca5640 1 -- 172.21.15.31:0/2154784029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774107890 msgr2=0x7f0774109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.226+0000 7f077bca5640 1 --2- 172.21.15.31:0/2154784029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774107890 0x7f0774109c80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f076c009900 tx=0x7f076c02a410 comp rx=0 tx=0).stop 2024-09-16T00:49:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.226+0000 7f077bca5640 1 -- 172.21.15.31:0/2154784029 shutdown_connections 2024-09-16T00:49:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.226+0000 7f077bca5640 1 --2- 172.21.15.31:0/2154784029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774107890 0x7f0774109c80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.226+0000 7f077bca5640 1 --2- 172.21.15.31:0/2154784029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0774104f60 0x7f0774107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.226+0000 7f077bca5640 1 --2- 172.21.15.31:0/2154784029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0774102630 0x7f0774104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.226+0000 7f077bca5640 1 -- 172.21.15.31:0/2154784029 >> 172.21.15.31:0/2154784029 conn(0x7f07740fc170 msgr2=0x7f07740fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.226+0000 7f077bca5640 1 -- 172.21.15.31:0/2154784029 shutdown_connections 2024-09-16T00:49:08.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.226+0000 7f077bca5640 1 -- 172.21.15.31:0/2154784029 wait complete. 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f077bca5640 1 Processor -- start 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f077bca5640 1 -- start start 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f077bca5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774102630 0x7f0774197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f077bca5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0774104f60 0x7f0774198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f077bca5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0774107890 0x7f077419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f077bca5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f077410d7b0 con 0x7f0774102630 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f077bca5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f077410d630 con 0x7f0774104f60 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f077bca5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f077410d930 con 0x7f0774107890 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f0779a41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774102630 0x7f0774197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f0779a41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774102630 0x7f0774197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54512/0 (socket says 172.21.15.31:54512) 2024-09-16T00:49:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f0779a41640 1 -- 172.21.15.31:0/4256411764 learned_addr learned my addr 172.21.15.31:0/4256411764 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f0779240640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0774104f60 0x7f0774198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.227+0000 7f077a242640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0774107890 0x7f077419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f0779a41640 1 -- 172.21.15.31:0/4256411764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0774107890 msgr2=0x7f077419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f0779a41640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0774107890 0x7f077419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f0779a41640 1 -- 172.21.15.31:0/4256411764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0774104f60 msgr2=0x7f0774198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f0779a41640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0774104f60 0x7f0774198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f0779a41640 1 -- 172.21.15.31:0/4256411764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077419fa50 con 0x7f0774102630 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f077a242640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0774107890 0x7f077419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f0779a41640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774102630 0x7f0774197e30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f077000cce0 tx=0x7f077000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f076affd640 1 -- 172.21.15.31:0/4256411764 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770004b20 con 0x7f0774102630 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f076affd640 1 -- 172.21.15.31:0/4256411764 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0770004cc0 con 0x7f0774102630 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077419fd40 con 0x7f0774102630 2024-09-16T00:49:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f076affd640 1 -- 172.21.15.31:0/4256411764 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077000f7d0 con 0x7f0774102630 2024-09-16T00:49:08.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.228+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07741a01d0 con 0x7f0774102630 2024-09-16T00:49:08.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.229+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07740fdc60 con 0x7f0774102630 2024-09-16T00:49:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.230+0000 7f076affd640 1 -- 172.21.15.31:0/4256411764 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f077000f970 con 0x7f0774102630 2024-09-16T00:49:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.231+0000 7f076affd640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07600a3250 0x7f07600a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.231+0000 7f0779240640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07600a3250 0x7f07600a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.231+0000 7f076affd640 1 -- 172.21.15.31:0/4256411764 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f07700cfdd0 con 0x7f0774102630 2024-09-16T00:49:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.231+0000 7f0779240640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07600a3250 0x7f07600a5710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0774199340 tx=0x7f076400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:08.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.235+0000 7f076affd640 1 -- 172.21.15.31:0/4256411764 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0770014030 con 0x7f0774102630 2024-09-16T00:49:08.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.412+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0774062ef0 con 0x7f0774102630 2024-09-16T00:49:08.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.414+0000 7f076affd640 1 -- 172.21.15.31:0/4256411764 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24766 (secure 0 0 0) 0x7f077009c6e0 con 0x7f0774102630 2024-09-16T00:49:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:49:08.416 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:40:59.575845+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:49:08.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07600a3250 msgr2=0x7f07600a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07600a3250 0x7f07600a5710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0774199340 tx=0x7f076400b410 comp rx=0 tx=0).stop 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774102630 msgr2=0x7f0774197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774102630 0x7f0774197e30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f077000cce0 tx=0x7f077000a620 comp rx=0 tx=0).stop 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 shutdown_connections 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f07600a3250 0x7f07600a5710 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0774107890 0x7f077419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0774104f60 0x7f0774198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 --2- 172.21.15.31:0/4256411764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0774102630 0x7f0774197e30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 >> 172.21.15.31:0/4256411764 conn(0x7f07740fc170 msgr2=0x7f0774102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 shutdown_connections 2024-09-16T00:49:08.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.416+0000 7f077bca5640 1 -- 172.21.15.31:0/4256411764 wait complete. 2024-09-16T00:49:08.429 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:34:12.067708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:38:49.806646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:38:51.945160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-16T00:38:58.457307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-16T00:49:08.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T00:49:08.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 -- 172.21.15.31:0/3967248105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c1025b0 msgr2=0x7f353c108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 --2- 172.21.15.31:0/3967248105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c1025b0 0x7f353c108e40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3534009ca0 tx=0x7f353402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 -- 172.21.15.31:0/3967248105 shutdown_connections 2024-09-16T00:49:08.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 --2- 172.21.15.31:0/3967248105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c1025b0 0x7f353c108e40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 --2- 172.21.15.31:0/3967248105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f353c101bf0 0x7f353c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 --2- 172.21.15.31:0/3967248105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f353c100a00 0x7f353c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 -- 172.21.15.31:0/3967248105 >> 172.21.15.31:0/3967248105 conn(0x7f353c0fc140 msgr2=0x7f353c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 -- 172.21.15.31:0/3967248105 shutdown_connections 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 -- 172.21.15.31:0/3967248105 wait complete. 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 Processor -- start 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.534+0000 7f354281e640 1 -- start start 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f354281e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c100a00 0x7f353c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f354281e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f353c101bf0 0x7f353c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f354281e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f353c1025b0 0x7f353c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f354281e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f353c10d7d0 con 0x7f353c100a00 2024-09-16T00:49:08.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f354281e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f353c10d650 con 0x7f353c101bf0 2024-09-16T00:49:08.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f354281e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f353c10d950 con 0x7f353c1025b0 2024-09-16T00:49:08.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c100a00 0x7f353c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c100a00 0x7f353c197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54530/0 (socket says 172.21.15.31:54530) 2024-09-16T00:49:08.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353bfff640 1 -- 172.21.15.31:0/728735788 learned_addr learned my addr 172.21.15.31:0/728735788 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:08.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f3540dbb640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f353c1025b0 0x7f353c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353b7fe640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f353c101bf0 0x7f353c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353bfff640 1 -- 172.21.15.31:0/728735788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f353c1025b0 msgr2=0x7f353c19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353bfff640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f353c1025b0 0x7f353c19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353bfff640 1 -- 172.21.15.31:0/728735788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f353c101bf0 msgr2=0x7f353c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353bfff640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f353c101bf0 0x7f353c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353bfff640 1 -- 172.21.15.31:0/728735788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353c19fa40 con 0x7f353c100a00 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353b7fe640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f353c101bf0 0x7f353c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.535+0000 7f353bfff640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c100a00 0x7f353c197e20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9c0 tx=0x7f352c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.536+0000 7f35397fa640 1 -- 172.21.15.31:0/728735788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c00cd80 con 0x7f353c100a00 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.536+0000 7f35397fa640 1 -- 172.21.15.31:0/728735788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c0035f0 con 0x7f353c100a00 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.536+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353c19fd30 con 0x7f353c100a00 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.536+0000 7f35397fa640 1 -- 172.21.15.31:0/728735788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c010690 con 0x7f353c100a00 2024-09-16T00:49:08.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.536+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f353c1a0210 con 0x7f353c100a00 2024-09-16T00:49:08.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.537+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f353c107de0 con 0x7f353c100a00 2024-09-16T00:49:08.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.538+0000 7f35397fa640 1 -- 172.21.15.31:0/728735788 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f352c0037a0 con 0x7f353c100a00 2024-09-16T00:49:08.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.539+0000 7f35397fa640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35240a32d0 0x7f35240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.539+0000 7f35397fa640 1 -- 172.21.15.31:0/728735788 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f352c0d0150 con 0x7f353c100a00 2024-09-16T00:49:08.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.539+0000 7f353b7fe640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35240a32d0 0x7f35240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.539+0000 7f353b7fe640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35240a32d0 0x7f35240a5790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f353c199330 tx=0x7f3530005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:08.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.542+0000 7f35397fa640 1 -- 172.21.15.31:0/728735788 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f352c09cb50 con 0x7f353c100a00 2024-09-16T00:49:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.719+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f353c062f30 con 0x7f353c100a00 2024-09-16T00:49:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.719+0000 7f35397fa640 1 -- 172.21.15.31:0/728735788 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f352c09c450 con 0x7f353c100a00 2024-09-16T00:49:08.721 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-16T00:49:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.721+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35240a32d0 msgr2=0x7f35240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.721+0000 7f354281e640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35240a32d0 0x7f35240a5790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f353c199330 tx=0x7f3530005f90 comp rx=0 tx=0).stop 2024-09-16T00:49:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.721+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c100a00 msgr2=0x7f353c197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.721+0000 7f354281e640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c100a00 0x7f353c197e20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9c0 tx=0x7f352c00ee80 comp rx=0 tx=0).stop 2024-09-16T00:49:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.722+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 shutdown_connections 2024-09-16T00:49:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.722+0000 7f354281e640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f35240a32d0 0x7f35240a5790 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.722+0000 7f354281e640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f353c1025b0 0x7f353c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.722+0000 7f354281e640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f353c101bf0 0x7f353c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.722+0000 7f354281e640 1 --2- 172.21.15.31:0/728735788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c100a00 0x7f353c197e20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.722+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 >> 172.21.15.31:0/728735788 conn(0x7f353c0fc140 msgr2=0x7f353c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.722+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 shutdown_connections 2024-09-16T00:49:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.722+0000 7f354281e640 1 -- 172.21.15.31:0/728735788 wait complete. 2024-09-16T00:49:08.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:08 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4256411764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:49:08.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T00:49:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 -- 172.21.15.31:0/1304728820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144101c20 msgr2=0x7fa1441020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 --2- 172.21.15.31:0/1304728820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144101c20 0x7fa1441020a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa138009900 tx=0x7fa13802a410 comp rx=0 tx=0).stop 2024-09-16T00:49:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 -- 172.21.15.31:0/1304728820 shutdown_connections 2024-09-16T00:49:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 --2- 172.21.15.31:0/1304728820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1441025e0 0x7fa144108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 --2- 172.21.15.31:0/1304728820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144101c20 0x7fa1441020a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 --2- 172.21.15.31:0/1304728820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa144100a30 0x7fa144100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 -- 172.21.15.31:0/1304728820 >> 172.21.15.31:0/1304728820 conn(0x7fa1440fc1b0 msgr2=0x7fa1440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 -- 172.21.15.31:0/1304728820 shutdown_connections 2024-09-16T00:49:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 -- 172.21.15.31:0/1304728820 wait complete. 2024-09-16T00:49:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.839+0000 7fa149c77640 1 Processor -- start 2024-09-16T00:49:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa149c77640 1 -- start start 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa149c77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144100a30 0x7fa1441049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa149c77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa144101c20 0x7fa144103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa149c77640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1441025e0 0x7fa144103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa149c77640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa14410c6b0 con 0x7fa144100a30 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa149c77640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa14410c530 con 0x7fa1441025e0 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa149c77640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa14410c830 con 0x7fa144101c20 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa1437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144100a30 0x7fa1441049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa1437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144100a30 0x7fa1441049c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54540/0 (socket says 172.21.15.31:54540) 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa1437fe640 1 -- 172.21.15.31:0/3598682293 learned_addr learned my addr 172.21.15.31:0/3598682293 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa143fff640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1441025e0 0x7fa144103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa142ffd640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa144101c20 0x7fa144103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa1437fe640 1 -- 172.21.15.31:0/3598682293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa144101c20 msgr2=0x7fa144103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa1437fe640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa144101c20 0x7fa144103000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa1437fe640 1 -- 172.21.15.31:0/3598682293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1441025e0 msgr2=0x7fa144103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa1437fe640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1441025e0 0x7fa144103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa1437fe640 1 -- 172.21.15.31:0/3598682293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa144103dd0 con 0x7fa144100a30 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.840+0000 7fa143fff640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1441025e0 0x7fa144103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.841+0000 7fa1437fe640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144100a30 0x7fa1441049c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa13400d950 tx=0x7fa13400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.841+0000 7fa140ff9640 1 -- 172.21.15.31:0/3598682293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134017070 con 0x7fa144100a30 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.841+0000 7fa140ff9640 1 -- 172.21.15.31:0/3598682293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa134009c30 con 0x7fa144100a30 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.841+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1441a2090 con 0x7fa144100a30 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.841+0000 7fa140ff9640 1 -- 172.21.15.31:0/3598682293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13400f600 con 0x7fa144100a30 2024-09-16T00:49:08.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.841+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1441a2590 con 0x7fa144100a30 2024-09-16T00:49:08.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.842+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1441a2850 con 0x7fa144100a30 2024-09-16T00:49:08.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.843+0000 7fa140ff9640 1 -- 172.21.15.31:0/3598682293 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa13401a070 con 0x7fa144100a30 2024-09-16T00:49:08.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.844+0000 7fa140ff9640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa12c0a3250 0x7fa12c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:08.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.844+0000 7fa142ffd640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa12c0a3250 0x7fa12c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:08.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.844+0000 7fa140ff9640 1 -- 172.21.15.31:0/3598682293 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fa1340cf850 con 0x7fa144100a30 2024-09-16T00:49:08.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.844+0000 7fa142ffd640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa12c0a3250 0x7fa12c0a5710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa1380096c0 tx=0x7fa138000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:08.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:08.847+0000 7fa140ff9640 1 -- 172.21.15.31:0/3598682293 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa134009de0 con 0x7fa144100a30 2024-09-16T00:49:08.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:08 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4256411764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:49:09.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.024+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1441a2a20 con 0x7fa144100a30 2024-09-16T00:49:09.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.024+0000 7fa140ff9640 1 -- 172.21.15.31:0/3598682293 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fa13409c160 con 0x7fa144100a30 2024-09-16T00:49:09.031 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-09-16T00:49:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa12c0a3250 msgr2=0x7fa12c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa12c0a3250 0x7fa12c0a5710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa1380096c0 tx=0x7fa138000bc0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144100a30 msgr2=0x7fa1441049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144100a30 0x7fa1441049c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa13400d950 tx=0x7fa13400de10 comp rx=0 tx=0).stop 2024-09-16T00:49:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 shutdown_connections 2024-09-16T00:49:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1441025e0 0x7fa144103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fa12c0a3250 0x7fa12c0a5710 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa144101c20 0x7fa144103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 --2- 172.21.15.31:0/3598682293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa144100a30 0x7fa1441049c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 >> 172.21.15.31:0/3598682293 conn(0x7fa1440fc1b0 msgr2=0x7fa14406a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:09.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 shutdown_connections 2024-09-16T00:49:09.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.026+0000 7fa149c77640 1 -- 172.21.15.31:0/3598682293 wait complete. 2024-09-16T00:49:09.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T00:49:09.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:08 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4256411764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:49:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.145+0000 7fdcd800e640 1 -- 172.21.15.31:0/3920236429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0101c20 msgr2=0x7fdcd01020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.145+0000 7fdcd800e640 1 --2- 172.21.15.31:0/3920236429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0101c20 0x7fdcd01020a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009900 tx=0x7fdcc002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.145+0000 7fdcd800e640 1 -- 172.21.15.31:0/3920236429 shutdown_connections 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.145+0000 7fdcd800e640 1 --2- 172.21.15.31:0/3920236429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcd01025e0 0x7fdcd0108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.145+0000 7fdcd800e640 1 --2- 172.21.15.31:0/3920236429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0101c20 0x7fdcd01020a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.145+0000 7fdcd800e640 1 --2- 172.21.15.31:0/3920236429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcd0100a30 0x7fdcd0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.145+0000 7fdcd800e640 1 -- 172.21.15.31:0/3920236429 >> 172.21.15.31:0/3920236429 conn(0x7fdcd00fc1b0 msgr2=0x7fdcd00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.145+0000 7fdcd800e640 1 -- 172.21.15.31:0/3920236429 shutdown_connections 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.145+0000 7fdcd800e640 1 -- 172.21.15.31:0/3920236429 wait complete. 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd800e640 1 Processor -- start 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd800e640 1 -- start start 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd800e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0100a30 0x7fdcd0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd800e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcd0101c20 0x7fdcd01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd800e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcd01025e0 0x7fdcd019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd800e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010d800 con 0x7fdcd0100a30 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd800e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010d680 con 0x7fdcd0101c20 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd800e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010d980 con 0x7fdcd01025e0 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd5daa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0100a30 0x7fdcd0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd5daa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0100a30 0x7fdcd0197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54546/0 (socket says 172.21.15.31:54546) 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd5daa640 1 -- 172.21.15.31:0/1906258137 learned_addr learned my addr 172.21.15.31:0/1906258137 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd55a9640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcd0101c20 0x7fdcd01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd65ab640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcd01025e0 0x7fdcd019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd5daa640 1 -- 172.21.15.31:0/1906258137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcd01025e0 msgr2=0x7fdcd019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd5daa640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcd01025e0 0x7fdcd019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd5daa640 1 -- 172.21.15.31:0/1906258137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcd0101c20 msgr2=0x7fdcd01983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd5daa640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcd0101c20 0x7fdcd01983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd5daa640 1 -- 172.21.15.31:0/1906258137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd019fac0 con 0x7fdcd0100a30 2024-09-16T00:49:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.146+0000 7fdcd5daa640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0100a30 0x7fdcd0197ea0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00d950 tx=0x7fdccc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.147+0000 7fdcc6ffd640 1 -- 172.21.15.31:0/1906258137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc017070 con 0x7fdcd0100a30 2024-09-16T00:49:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.147+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd019fdb0 con 0x7fdcd0100a30 2024-09-16T00:49:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.147+0000 7fdcc6ffd640 1 -- 172.21.15.31:0/1906258137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdccc009c30 con 0x7fdcd0100a30 2024-09-16T00:49:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.147+0000 7fdcc6ffd640 1 -- 172.21.15.31:0/1906258137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc00f600 con 0x7fdcd0100a30 2024-09-16T00:49:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.147+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd01a0210 con 0x7fdcd0100a30 2024-09-16T00:49:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.148+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd0107dc0 con 0x7fdcd0100a30 2024-09-16T00:49:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.149+0000 7fdcc6ffd640 1 -- 172.21.15.31:0/1906258137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fdccc01a070 con 0x7fdcd0100a30 2024-09-16T00:49:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.150+0000 7fdcc6ffd640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcbc0a3250 0x7fdcbc0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.150+0000 7fdcd55a9640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcbc0a3250 0x7fdcbc0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.150+0000 7fdcc6ffd640 1 -- 172.21.15.31:0/1906258137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fdccc0cf7d0 con 0x7fdcd0100a30 2024-09-16T00:49:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.150+0000 7fdcd55a9640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcbc0a3250 0x7fdcbc0a5710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdcd01993b0 tx=0x7fdcc00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.154+0000 7fdcc6ffd640 1 -- 172.21.15.31:0/1906258137 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdccc009de0 con 0x7fdcd0100a30 2024-09-16T00:49:09.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.335+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdcd0062e90 con 0x7fdcd0100a30 2024-09-16T00:49:09.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.336+0000 7fdcc6ffd640 1 -- 172.21.15.31:0/1906258137 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fdccc09c140 con 0x7fdcd0100a30 2024-09-16T00:49:09.337 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-09-16T00:49:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcbc0a3250 msgr2=0x7fdcbc0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcbc0a3250 0x7fdcbc0a5710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdcd01993b0 tx=0x7fdcc00315b0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0100a30 msgr2=0x7fdcd0197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0100a30 0x7fdcd0197ea0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00d950 tx=0x7fdccc00de10 comp rx=0 tx=0).stop 2024-09-16T00:49:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 shutdown_connections 2024-09-16T00:49:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fdcbc0a3250 0x7fdcbc0a5710 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcd01025e0 0x7fdcd019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcd0101c20 0x7fdcd01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 --2- 172.21.15.31:0/1906258137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd0100a30 0x7fdcd0197ea0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 >> 172.21.15.31:0/1906258137 conn(0x7fdcd00fc1b0 msgr2=0x7fdcd0104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 shutdown_connections 2024-09-16T00:49:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.338+0000 7fdcd800e640 1 -- 172.21.15.31:0/1906258137 wait complete. 2024-09-16T00:49:09.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-16T00:49:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.454+0000 7fc279bf6640 1 -- 172.21.15.31:0/1952621092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 msgr2=0x7fc274109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.454+0000 7fc279bf6640 1 --2- 172.21.15.31:0/1952621092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 0x7fc274109c80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009ca0 tx=0x7fc26c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.454+0000 7fc279bf6640 1 -- 172.21.15.31:0/1952621092 shutdown_connections 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.454+0000 7fc279bf6640 1 --2- 172.21.15.31:0/1952621092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 0x7fc274109c80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.454+0000 7fc279bf6640 1 --2- 172.21.15.31:0/1952621092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc274104ee0 0x7fc2741072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.454+0000 7fc279bf6640 1 --2- 172.21.15.31:0/1952621092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2741025b0 0x7fc2741049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.454+0000 7fc279bf6640 1 -- 172.21.15.31:0/1952621092 >> 172.21.15.31:0/1952621092 conn(0x7fc2740fc1b0 msgr2=0x7fc2740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.455+0000 7fc279bf6640 1 -- 172.21.15.31:0/1952621092 shutdown_connections 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.455+0000 7fc279bf6640 1 -- 172.21.15.31:0/1952621092 wait complete. 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.455+0000 7fc279bf6640 1 Processor -- start 2024-09-16T00:49:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.455+0000 7fc279bf6640 1 -- start start 2024-09-16T00:49:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.455+0000 7fc279bf6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2741025b0 0x7fc274197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.455+0000 7fc279bf6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc274104ee0 0x7fc2741983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.455+0000 7fc279bf6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 0x7fc27419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.455+0000 7fc279bf6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc27410d7d0 con 0x7fc274107810 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.455+0000 7fc279bf6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc27410d650 con 0x7fc274104ee0 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc279bf6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc27410d950 con 0x7fc2741025b0 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc273fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 0x7fc27419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc273fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 0x7fc27419f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54566/0 (socket says 172.21.15.31:54566) 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc273fff640 1 -- 172.21.15.31:0/72625920 learned_addr learned my addr 172.21.15.31:0/72625920 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc2737fe640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2741025b0 0x7fc274197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc272ffd640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc274104ee0 0x7fc2741983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc273fff640 1 -- 172.21.15.31:0/72625920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2741025b0 msgr2=0x7fc274197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc273fff640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2741025b0 0x7fc274197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc273fff640 1 -- 172.21.15.31:0/72625920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc274104ee0 msgr2=0x7fc2741983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc273fff640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc274104ee0 0x7fc2741983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc273fff640 1 -- 172.21.15.31:0/72625920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27419fa80 con 0x7fc274107810 2024-09-16T00:49:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc273fff640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 0x7fc27419f410 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009bf0 tx=0x7fc26c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc270ff9640 1 -- 172.21.15.31:0/72625920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c003b70 con 0x7fc274107810 2024-09-16T00:49:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27419fd10 con 0x7fc274107810 2024-09-16T00:49:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc270ff9640 1 -- 172.21.15.31:0/72625920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26c003d10 con 0x7fc274107810 2024-09-16T00:49:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc270ff9640 1 -- 172.21.15.31:0/72625920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c02fb80 con 0x7fc274107810 2024-09-16T00:49:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.456+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2741a0200 con 0x7fc274107810 2024-09-16T00:49:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.457+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2740fdca0 con 0x7fc274107810 2024-09-16T00:49:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.459+0000 7fc270ff9640 1 -- 172.21.15.31:0/72625920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fc26c02fd20 con 0x7fc274107810 2024-09-16T00:49:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.459+0000 7fc270ff9640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc25c0a3250 0x7fc25c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.459+0000 7fc2737fe640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc25c0a3250 0x7fc25c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.459+0000 7fc270ff9640 1 -- 172.21.15.31:0/72625920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fc26c0f0ca0 con 0x7fc274107810 2024-09-16T00:49:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.460+0000 7fc2737fe640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc25c0a3250 0x7fc25c0a5710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc264009bd0 tx=0x7fc264005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:09.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.463+0000 7fc270ff9640 1 -- 172.21.15.31:0/72625920 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc26c0bd730 con 0x7fc274107810 2024-09-16T00:49:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.638+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc274062e90 con 0x7fc274107810 2024-09-16T00:49:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.638+0000 7fc270ff9640 1 -- 172.21.15.31:0/72625920 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fc26c0bd030 con 0x7fc274107810 2024-09-16T00:49:09.641 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-09-16T00:49:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc25c0a3250 msgr2=0x7fc25c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc25c0a3250 0x7fc25c0a5710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc264009bd0 tx=0x7fc264005f90 comp rx=0 tx=0).stop 2024-09-16T00:49:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 msgr2=0x7fc27419f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 0x7fc27419f410 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009bf0 tx=0x7fc26c004ab0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 shutdown_connections 2024-09-16T00:49:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc274107810 0x7fc27419f410 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc274104ee0 0x7fc2741983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fc25c0a3250 0x7fc25c0a5710 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 --2- 172.21.15.31:0/72625920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2741025b0 0x7fc274197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 >> 172.21.15.31:0/72625920 conn(0x7fc2740fc1b0 msgr2=0x7fc274102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 shutdown_connections 2024-09-16T00:49:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.640+0000 7fc279bf6640 1 -- 172.21.15.31:0/72625920 wait complete. 2024-09-16T00:49:09.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-16T00:49:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.756+0000 7f62aa5a2640 1 -- 172.21.15.31:0/2171220909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4102420 msgr2=0x7f62a4108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.756+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/2171220909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4102420 0x7f62a4108cb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f629c009ca0 tx=0x7f629c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.756+0000 7f62aa5a2640 1 -- 172.21.15.31:0/2171220909 shutdown_connections 2024-09-16T00:49:09.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.756+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/2171220909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4102420 0x7f62a4108cb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.756+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/2171220909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62a4101a60 0x7f62a4101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.756+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/2171220909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62a4100870 0x7f62a4100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.756+0000 7f62aa5a2640 1 -- 172.21.15.31:0/2171220909 >> 172.21.15.31:0/2171220909 conn(0x7f62a40fbfb0 msgr2=0x7f62a40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:09.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.756+0000 7f62aa5a2640 1 -- 172.21.15.31:0/2171220909 shutdown_connections 2024-09-16T00:49:09.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.756+0000 7f62aa5a2640 1 -- 172.21.15.31:0/2171220909 wait complete. 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62aa5a2640 1 Processor -- start 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62aa5a2640 1 -- start start 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62aa5a2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62a4100870 0x7f62a406acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62aa5a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4101a60 0x7f62a419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62aa5a2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62a4102420 0x7f62a419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62aa5a2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f62a410c4d0 con 0x7f62a4101a60 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62aa5a2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f62a410c350 con 0x7f62a4102420 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62aa5a2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62a410c650 con 0x7f62a4100870 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62a8d9f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4101a60 0x7f62a419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62a8d9f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4101a60 0x7f62a419c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54580/0 (socket says 172.21.15.31:54580) 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62a8d9f640 1 -- 172.21.15.31:0/931867005 learned_addr learned my addr 172.21.15.31:0/931867005 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:09.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.757+0000 7f62a95a0640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62a4100870 0x7f62a406acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62a9da1640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62a4102420 0x7f62a419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62a8d9f640 1 -- 172.21.15.31:0/931867005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62a4100870 msgr2=0x7f62a406acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62a8d9f640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62a4100870 0x7f62a406acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62a8d9f640 1 -- 172.21.15.31:0/931867005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62a4102420 msgr2=0x7f62a419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62a8d9f640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62a4102420 0x7f62a419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62a8d9f640 1 -- 172.21.15.31:0/931867005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a419f750 con 0x7f62a4101a60 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62a9da1640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62a4102420 0x7f62a419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62a8d9f640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4101a60 0x7f62a419c840 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f629800c920 tx=0x7f629800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62967fc640 1 -- 172.21.15.31:0/931867005 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298007c70 con 0x7f62a4101a60 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a419fa40 con 0x7f62a4101a60 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62967fc640 1 -- 172.21.15.31:0/931867005 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6298007e10 con 0x7f62a4101a60 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62967fc640 1 -- 172.21.15.31:0/931867005 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298018890 con 0x7f62a4101a60 2024-09-16T00:49:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.758+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a419ff90 con 0x7f62a4101a60 2024-09-16T00:49:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.759+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a4107b90 con 0x7f62a4101a60 2024-09-16T00:49:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.760+0000 7f62967fc640 1 -- 172.21.15.31:0/931867005 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6298018a50 con 0x7f62a4101a60 2024-09-16T00:49:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.761+0000 7f62967fc640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62900a3250 0x7f62900a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.761+0000 7f62967fc640 1 -- 172.21.15.31:0/931867005 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f62980d00a0 con 0x7f62a4101a60 2024-09-16T00:49:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.761+0000 7f62a95a0640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62900a3250 0x7f62900a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.761+0000 7f62a95a0640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62900a3250 0x7f62900a5710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f62a41018c0 tx=0x7f62a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.764+0000 7f62967fc640 1 -- 172.21.15.31:0/931867005 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6298014030 con 0x7f62a4101a60 2024-09-16T00:49:09.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.988+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f62a4062df0 con 0x7f62a4101a60 2024-09-16T00:49:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:09.996+0000 7f62967fc640 1 -- 172.21.15.31:0/931867005 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+742338 (secure 0 0 0) 0x7f629809caa0 con 0x7f62a4101a60 2024-09-16T00:49:09.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:09 smithi081 ceph-mon[35539]: pgmap v596: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:09.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:09 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/728735788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:49:09.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:09 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3598682293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:49:09.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:09 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1906258137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:49:09.998 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:09 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/72625920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:49:10.008 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:49:10.008 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:49:10.008 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 18, 2024-09-16T00:49:10.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.008 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 14150, 2024-09-16T00:49:10.008 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2024-09-16T00:49:10.008 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2024-09-16T00:49:10.008 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:49:10.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 502885538 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 502885538 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6801/502885538", 2024-09-16T00:49:10.009 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2024-09-16T00:31:57.569290+0000", 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540701547738038271, 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24109, 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "z", 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.010 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.014 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-16T00:49:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-16T00:49:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.023 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-16T00:49:10.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.024 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-16T00:49:10.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.028 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.029 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.029 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-16T00:49:10.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:49:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:10.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-16T00:49:10.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-16T00:49:10.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.039 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.045 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-16T00:49:10.046 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-16T00:49:10.046 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-16T00:49:10.046 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-16T00:49:10.046 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-16T00:49:10.046 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:49:10.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.048 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-16T00:49:10.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:49:10.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:49:10.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-16T00:49:10.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:49:10.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.057 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:49:10.058 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-16T00:49:10.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.063 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:49:10.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:49:10.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-16T00:49:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-16T00:49:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-16T00:49:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-16T00:49:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-16T00:49:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-16T00:49:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-16T00:49:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-16T00:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-16T00:49:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-16T00:49:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-16T00:49:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-16T00:49:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:49:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:49:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:49:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:49:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:49:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-16T00:49:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:49:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-16T00:49:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:49:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:49:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:49:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-16T00:49:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:49:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:49:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:49:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:49:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:49:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-16T00:49:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-16T00:49:10.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-16T00:49:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:49:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.186 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.189 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.190 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.191 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:49:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:49:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-16T00:49:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-16T00:49:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:49:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:49:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-16T00:49:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:49:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:49:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-16T00:49:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-16T00:49:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:49:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:49:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-16T00:49:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:49:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-16T00:49:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:49:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:49:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:49:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-16T00:49:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-16T00:49:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-16T00:49:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-16T00:49:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-16T00:49:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-16T00:49:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-16T00:49:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-16T00:49:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-16T00:49:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-16T00:49:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-16T00:49:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-16T00:49:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-16T00:49:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-16T00:49:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-16T00:49:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-16T00:49:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-16T00:49:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-16T00:49:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-16T00:49:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-16T00:49:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-16T00:49:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-16T00:49:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-16T00:49:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-16T00:49:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-16T00:49:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-16T00:49:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-16T00:49:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-16T00:49:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-16T00:49:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:49:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-16T00:49:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-16T00:49:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:49:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-16T00:49:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-16T00:49:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-16T00:49:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-16T00:49:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-16T00:49:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24116, 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-16T00:49:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:49:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-16T00:49:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:49:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:49:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:49:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-16T00:49:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-16T00:49:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:49:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-16T00:49:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-16T00:49:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-16T00:49:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:49:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:49:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:49:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-16T00:49:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-16T00:49:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:49:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-16T00:49:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-16T00:49:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-16T00:49:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:49:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:49:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:49:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:49:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.636 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.637 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:49:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:49:10.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:49:10.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.669 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-16T00:49:10.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-16T00:49:10.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-16T00:49:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.684 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.689 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-16T00:49:10.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-16T00:49:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.700 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.705 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.707 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.710 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-16T00:49:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.713 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.716 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.719 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.721 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-16T00:49:10.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.726 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.731 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.732 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-16T00:49:10.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.755 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-16T00:49:10.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-16T00:49:10.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-16T00:49:10.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.761 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.764 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.766 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-16T00:49:10.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.780 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-16T00:49:10.785 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-16T00:49:10.786 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.787 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.790 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.792 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.793 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.794 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.796 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.797 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.798 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.799 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:49:10.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-16T00:49:10.806 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:49:10.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.812 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-16T00:49:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:49:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:49:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-16T00:49:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-16T00:49:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-16T00:49:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-16T00:49:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-16T00:49:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-16T00:49:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-16T00:49:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-16T00:49:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:49:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-16T00:49:10.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard", 2024-09-16T00:49:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "restful" 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:10.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.971 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.972 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.974 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:49:10.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-16T00:49:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:10.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-16T00:49:10.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-16T00:49:10.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.991 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.993 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:10.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:10.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:10.999 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-16T00:49:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.007 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:49:11.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:49:11.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-16T00:49:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:49:11.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-16T00:49:11.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-16T00:49:11.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.030 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.031 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:49:11.034 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-16T00:49:11.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-16T00:49:11.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-16T00:49:11.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-16T00:49:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.043 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.047 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-16T00:49:11.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.049 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:49:11.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.054 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-16T00:49:11.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-16T00:49:11.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-16T00:49:11.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-16T00:49:11.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.072 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.080 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.081 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-16T00:49:11.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-16T00:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:49:11.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:49:11.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.094 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:49:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:49:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:49:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:49:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:49:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:49:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:49:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:49:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:49:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-16T00:49:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:49:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-16T00:49:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-16T00:49:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-16T00:49:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-16T00:49:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:49:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-16T00:49:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:49:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:49:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:49:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.214 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-16T00:49:11.216 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:49:11.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.218 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.228 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.229 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-16T00:49:11.240 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.240 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.240 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.240 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.241 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-16T00:49:11.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.258 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.260 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.261 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.266 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.268 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-16T00:49:11.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.278 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.282 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-16T00:49:11.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.288 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.289 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.290 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.297 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-16T00:49:11.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-16T00:49:11.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:49:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-16T00:49:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:49:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-16T00:49:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-16T00:49:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-16T00:49:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:49:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:49:11.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: "services": { 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard": "https://172.21.15.31:8443/" 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2024-09-16T00:49:11.436 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:49:11.437 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:49:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:49:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "squid": [ 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:49:11.440 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 3, 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:49:11.441 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3644001294 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2675788865 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:49:11.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2230492820 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-16T00:49:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:49:11.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:49:11.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:49:11.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:49:11.444 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3103400321 2024-09-16T00:49:11.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:49:11.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:11.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:49:11.444 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:49:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62900a3250 msgr2=0x7f62900a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62900a3250 0x7f62900a5710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f62a41018c0 tx=0x7f62a0005f90 comp rx=0 tx=0).stop 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4101a60 msgr2=0x7f62a419c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4101a60 0x7f62a419c840 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f629800c920 tx=0x7f629800cde0 comp rx=0 tx=0).stop 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 shutdown_connections 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62a4102420 0x7f62a419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62a4101a60 0x7f62a419c840 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f62900a3250 0x7f62900a5710 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 --2- 172.21.15.31:0/931867005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62a4100870 0x7f62a406acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 >> 172.21.15.31:0/931867005 conn(0x7f62a40fbfb0 msgr2=0x7f62a4104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 shutdown_connections 2024-09-16T00:49:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:10.001+0000 7f62aa5a2640 1 -- 172.21.15.31:0/931867005 wait complete. 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:10 smithi081 ceph-mon[35539]: pgmap v597: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:10 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/931867005' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:09 smithi031 ceph-mon[30956]: pgmap v596: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:09 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/728735788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:09 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3598682293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:09 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1906258137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:09 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/72625920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:09 smithi102 ceph-mon[35364]: pgmap v596: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:09 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/728735788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:49:11.447 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:09 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3598682293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:49:11.448 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:09 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1906258137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:49:11.448 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:09 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/72625920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:49:11.448 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:10 smithi102 ceph-mon[35364]: pgmap v597: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:11.448 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:10 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/931867005' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:49:11.509 DEBUG:tasks.check_counter:Getting stats from a 2024-09-16T00:49:11.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-16T00:49:11.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:10 smithi031 ceph-mon[30956]: pgmap v597: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:11.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:10 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/931867005' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 2992, 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.003469294, 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000044462, 2024-09-16T00:49:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002378145, 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.004213245, 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.001404415 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 174, 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 171, 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 466896, 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 104528, 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.025867413, 2024-09-16T00:49:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.011188834, 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.017818846, 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 170, 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.010980245, 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000064589 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 466896, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 104528 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 130, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 101360, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.010674195, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.008444066, 2024-09-16T00:49:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000966482, 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 128, 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.008555255, 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000066837 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 101360 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148984, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 175, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 442303, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 175, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 175, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 442303, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:12.294 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:49:13.055 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-16T00:49:13.055 DEBUG:tasks.check_counter:Getting stats from d 2024-09-16T00:49:13.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-16T00:49:13.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:12 smithi081 ceph-mon[35539]: pgmap v598: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:13.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:12 smithi102 ceph-mon[35364]: pgmap v598: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:13.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:12 smithi031 ceph-mon[30956]: pgmap v598: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:13.811 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 180, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 177, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 468528, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 108368, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.028801964, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.012065057, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.019549938, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 176, 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.009090698, 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000051651 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 468528, 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 108368 2024-09-16T00:49:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 134, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 102064, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.010924392, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.008385586, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001162781, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 132, 2024-09-16T00:49:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.005434307, 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000041168 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 102064 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2024-09-16T00:49:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 2992, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.002367553, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000033910, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002347553, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.004147715, 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.001382571 2024-09-16T00:49:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 149007, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 181, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 443083, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 181, 2024-09-16T00:49:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 181, 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 443083, 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:13.823 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:49:14.535 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-16T00:49:14.535 DEBUG:tasks.check_counter:Getting stats from g 2024-09-16T00:49:14.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-16T00:49:14.801 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:15.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:14 smithi031 ceph-mon[30956]: pgmap v599: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-16T00:49:15.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:14 smithi081 ceph-mon[35539]: pgmap v599: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-16T00:49:15.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:14 smithi102 ceph-mon[35364]: pgmap v599: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-16T00:49:15.346 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:49:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2024-09-16T00:49:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2024-09-16T00:49:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 2992, 2024-09-16T00:49:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2024-09-16T00:49:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.003605393, 2024-09-16T00:49:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000039854, 2024-09-16T00:49:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002559887, 2024-09-16T00:49:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2024-09-16T00:49:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.004640211, 2024-09-16T00:49:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.001546737 2024-09-16T00:49:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T00:49:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:49:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-09-16T00:49:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 131, 2024-09-16T00:49:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:49:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 101536, 2024-09-16T00:49:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.010854947, 2024-09-16T00:49:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.008341945, 2024-09-16T00:49:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001294497, 2024-09-16T00:49:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 129, 2024-09-16T00:49:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.013816993, 2024-09-16T00:49:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000107108 2024-09-16T00:49:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:49:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 101536 2024-09-16T00:49:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 176, 2024-09-16T00:49:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 173, 2024-09-16T00:49:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 467440, 2024-09-16T00:49:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 105808, 2024-09-16T00:49:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.024290955, 2024-09-16T00:49:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.011616579, 2024-09-16T00:49:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.016173197, 2024-09-16T00:49:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 172, 2024-09-16T00:49:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.011399674, 2024-09-16T00:49:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000066277 2024-09-16T00:49:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 467440, 2024-09-16T00:49:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 105808 2024-09-16T00:49:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-16T00:49:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-16T00:49:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-16T00:49:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-16T00:49:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-16T00:49:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148967, 2024-09-16T00:49:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 29, 2024-09-16T00:49:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-16T00:49:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-16T00:49:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-16T00:49:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 177, 2024-09-16T00:49:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 442563, 2024-09-16T00:49:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 177, 2024-09-16T00:49:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:49:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:49:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 177, 2024-09-16T00:49:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 442563, 2024-09-16T00:49:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:49:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:15.371 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:49:16.067 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-16T00:49:16.067 DEBUG:tasks.check_counter:Getting stats from j 2024-09-16T00:49:16.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-16T00:49:16.328 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 185, 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 180, 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 471792, 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 109456, 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.030595857, 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.012006936, 2024-09-16T00:49:16.831 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.021097840, 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 180, 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.016631860, 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000092399 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.832 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 471792, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 109456 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 135, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 102240, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.010620106, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.007835440, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001200339, 2024-09-16T00:49:16.833 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 133, 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.005734298, 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000043115 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:49:16.834 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 102240 2024-09-16T00:49:16.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.835 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:16.835 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-09-16T00:49:16.835 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-09-16T00:49:16.835 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:49:16.835 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.000281611, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000208757, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:16.836 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:16.837 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:16.837 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:16.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.837 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:49:16.837 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:49:16.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.837 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:16.838 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:16.839 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:16.840 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 149007, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:16.841 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-16T00:49:16.842 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 182, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 443213, 2024-09-16T00:49:16.843 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 182, 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 182, 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 443213, 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:49:16.844 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:49:17.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:16 smithi031 ceph-mon[30956]: pgmap v600: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:17.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:16 smithi081 ceph-mon[35539]: pgmap v600: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:17.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:16 smithi102 ceph-mon[35364]: pgmap v600: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:17.507 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-16T00:49:17.507 DEBUG:tasks.check_counter:Getting stats from b 2024-09-16T00:49:17.507 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-16T00:49:17.805 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:49:18.386 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-16T00:49:18.386 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:18.386 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 89127, 2024-09-16T00:49:18.386 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 89129, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 28479433, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 274743085, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 11, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 4, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 6.778575671, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 2.585964712, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 1.189869288, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 2.611680276, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 89127, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 40.925003237, 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000459176 2024-09-16T00:49:18.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000161, 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000080 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 64226, 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 64354, 2024-09-16T00:49:18.388 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 24642597, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 280459411, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 103, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 5.368298426, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 2.144118188, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.919311179, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 1.984287279, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 64353, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 37.027853951, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000575386 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:18.389 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 96, 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000006649, 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000069 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 317104 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 479634, 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 681150, 2024-09-16T00:49:18.390 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 156161539, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 447010637, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 32, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 21.915818927, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 9.633806029, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 21.295559386, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 1.945973172, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 681149, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 61.286831034, 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000089975 2024-09-16T00:49:18.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 863817, 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.138898157, 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000160 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 556896, 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 156960 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-09-16T00:49:18.392 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 36012, 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 263.543821305, 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.007318222 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 48793, 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 31.236605488, 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000640186 2024-09-16T00:49:18.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 253830, 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 253830, 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 253830, 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 13425.695765052, 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.052892470 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 0, 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2024-09-16T00:49:18.394 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 1904, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 128, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 16, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 1874, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 1299, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 575, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 0, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 17, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 0, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 3, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 3, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 2, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 425234, 2024-09-16T00:49:18.395 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 354910, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 2, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-09-16T00:49:18.396 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 99103, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 4, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 234, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 0, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 99103, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 728, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 11, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 717, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 98375, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:49:18.397 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 105495, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 114966, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 7, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 149065, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 39838, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 7729, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 77349 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2024-09-16T00:49:18.398 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 1, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 38311, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 38310, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-09-16T00:49:18.399 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 6, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 74066, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 11, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 0, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 151, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 251467, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 150729, 2024-09-16T00:49:18.400 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 147481, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 103986, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 3248, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 318, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 190, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 185, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 133, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 9, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 5, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 351994489, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 541263040, 2024-09-16T00:49:18.401 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 14188, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 934.339110800, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.065854180 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 24, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 38345, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 38321, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 19, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 1198, 2024-09-16T00:49:18.402 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 1179, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 1874, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 84787, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 82913, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 3, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 57379, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 57376, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 180636, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 231712 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 258052, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 0, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 253830, 2024-09-16T00:49:18.403 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 17, 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 0, 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 38172, 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1116.903553989, 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.029259759 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3229, 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1184.584671075, 2024-09-16T00:49:18.404 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.366858058 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-09-16T00:49:18.405 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 129546, 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 8941.315995891, 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.069020394 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000361541, 2024-09-16T00:49:18.406 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000180770 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-09-16T00:49:18.407 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.408 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.408 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.408 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 146, 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 47.959554686, 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.328490100 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.409 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 21582, 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1084.571813188, 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.050253535 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1022, 2024-09-16T00:49:18.410 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.919731190, 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001878406 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 37794, 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 997.089746302, 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.026382223 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-09-16T00:49:18.411 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 140, 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 15.694451332, 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.112103223 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 9, 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.514012343, 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.057112482 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.413 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-09-16T00:49:18.414 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 8, 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.022968420, 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002871052 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3, 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.001761732, 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000587244 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 22177, 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 35.117143363, 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001583493 2024-09-16T00:49:18.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 1, 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 3, 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 2, 2024-09-16T00:49:18.416 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 1, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 55607, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 55607, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 494, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:18.417 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-09-16T00:49:18.419 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 288568, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 105, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 440, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 5, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:18.420 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 8880, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 169, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 1504474, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 21094, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-09-16T00:49:18.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 223238, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 592620292, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 31, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 223238, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 223238, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 4286.721363785, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.019202471 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-09-16T00:49:18.422 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 223238, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 269055 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 223238, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 99715, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 123523, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 720, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 27800, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 601, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 14779, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 278, 2024-09-16T00:49:18.423 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 128, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 78229, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 99114, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 41764, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-09-16T00:49:18.424 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 7, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-09-16T00:49:18.425 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 91, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 10, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 12, 2024-09-16T00:49:18.426 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 12, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 4515, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 0, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 1120 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-09-16T00:49:18.427 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 74113 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 55, 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 26, 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 77410, 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 38310, 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:49:18.428 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 632983, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 152293593, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 632983, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 632983, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 152293593, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.429 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 223238, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 596795898, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 223238, 2024-09-16T00:49:18.430 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 596795898, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 223238, 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 223238, 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 223238, 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 223238, 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:49:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 38310, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 4175606, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 38310, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 591, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 4175606, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x560ddc33a960": { 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 251467, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 537068736, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 251467, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 14188, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 537068736, 2024-09-16T00:49:18.434 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:49:18.435 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:18.435 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:18.435 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:18.435 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:18.435 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:18.435 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-16T00:49:19.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:18 smithi081 ceph-mon[35539]: pgmap v601: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:49:19.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:18 smithi102 ceph-mon[35364]: pgmap v601: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:49:19.307 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=128 2024-09-16T00:49:19.307 DEBUG:tasks.check_counter:Getting stats from e 2024-09-16T00:49:19.307 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-16T00:49:19.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:18 smithi031 ceph-mon[30956]: pgmap v601: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:49:19.563 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:49:20.094 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-16T00:49:20.094 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:20.094 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 181, 2024-09-16T00:49:20.094 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 178, 2024-09-16T00:49:20.094 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 468800, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 109008, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.023110183, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.010056546, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.016403214, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 177, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.008398094, 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000047446 2024-09-16T00:49:20.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 468800, 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 109008 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 5, 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 137, 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 3152, 2024-09-16T00:49:20.096 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 102688, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.009274473, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.005255450, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.002786696, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 136, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.010586433, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000077841 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:20.097 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 102688 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 0, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 2, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 0, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:20.098 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.000296186, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.000236414, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:20.099 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:20.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.100 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:49:20.100 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:49:20.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.101 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-09-16T00:49:20.101 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 0, 2024-09-16T00:49:20.101 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-09-16T00:49:20.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.101 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-09-16T00:49:20.102 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:20.102 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:20.103 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:20.104 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:20.105 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:20.105 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:20.105 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:20.105 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-09-16T00:49:20.105 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:20.105 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 149007, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 30, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:20.106 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 0, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 2, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:20.107 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 182, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 443213, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 182, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 182, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 443213, 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:49:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:20.109 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:20.109 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:20.109 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:20.109 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:20.109 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-16T00:49:20.818 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-09-16T00:49:20.818 DEBUG:tasks.check_counter:Getting stats from h 2024-09-16T00:49:20.819 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-16T00:49:21.066 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:49:21.187 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:20 smithi081 ceph-mon[35539]: pgmap v602: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-16T00:49:21.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:20 smithi102 ceph-mon[35364]: pgmap v602: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-16T00:49:21.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:20 smithi031 ceph-mon[30956]: pgmap v602: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-16T00:49:21.579 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-16T00:49:21.579 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:21.579 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 179, 2024-09-16T00:49:21.579 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 176, 2024-09-16T00:49:21.579 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 468256, 2024-09-16T00:49:21.579 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 107728, 2024-09-16T00:49:21.579 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.023318072, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.010101666, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.016829944, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 175, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.008487374, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000048499 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:21.580 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 468256, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 107728 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 134, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 102064, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:21.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.007622054, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.005399951, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.000955555, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 132, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.006332028, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000047969 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.582 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 102064 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 4, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 4, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 2992, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 448, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.002311317, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.000041095, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.002290435, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:21.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3, 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.004055308, 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001351769 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.584 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T00:49:21.585 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:49:21.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.585 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-09-16T00:49:21.586 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 0, 2024-09-16T00:49:21.586 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-09-16T00:49:21.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.586 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-09-16T00:49:21.586 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:21.586 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:21.586 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:21.586 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:21.586 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:21.587 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:21.588 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 148984, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 30, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:21.589 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 2, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:21.590 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 180, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 442953, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 180, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 180, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 442953, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:21.591 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:21.592 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:21.592 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:21.592 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:21.592 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-16T00:49:22.244 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.h 2024-09-16T00:49:22.245 DEBUG:tasks.check_counter:Getting stats from k 2024-09-16T00:49:22.245 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-16T00:49:22.490 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:49:23.004 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 188, 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 183, 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 472608, 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 111376, 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.026280653, 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.010119819, 2024-09-16T00:49:23.005 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.019568616, 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 183, 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.012707513, 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000069439 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 472608, 2024-09-16T00:49:23.006 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 111376 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 137, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 102592, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.007446187, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.005260239, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.000909882, 2024-09-16T00:49:23.007 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 135, 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.006846578, 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000050715 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 102592 2024-09-16T00:49:23.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 0, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 2, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 0, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.000302111, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.000230158, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:23.009 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 0, 2024-09-16T00:49:23.010 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:23.011 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:23.012 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:23.013 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 148967, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 29, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:23.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 2, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:49:23.015 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 185, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 443603, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 185, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 185, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 443603, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:49:23.016 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:49:23.017 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:23.017 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:49:23.017 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-16T00:49:23.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:22 smithi081 ceph-mon[35539]: pgmap v603: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:23.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:22 smithi102 ceph-mon[35364]: pgmap v603: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:22 smithi031 ceph-mon[30956]: pgmap v603: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:23.704 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-16T00:49:23.704 DEBUG:tasks.check_counter:Getting stats from c 2024-09-16T00:49:23.704 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-16T00:49:23.961 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:49:24.481 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T00:49:24.481 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:24.481 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 4, 2024-09-16T00:49:24.481 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 4, 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 2992, 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 448, 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.003150085, 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000032513, 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.002156161, 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:24.482 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3, 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.003961562, 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001320520 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:24.483 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 181, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 178, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 468800, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 109008, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.025204104, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.011284127, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.019165257, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:24.484 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 177, 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.009536942, 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000053881 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 468800, 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 109008 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.485 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 1, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 135, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 102240, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.008170401, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.005873942, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000818299, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 133, 2024-09-16T00:49:24.486 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.008643952, 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000064992 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 102240 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-16T00:49:24.487 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:24.488 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:24.489 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:24.490 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:24.491 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:24.491 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:24.491 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148984, 2024-09-16T00:49:24.491 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 30, 2024-09-16T00:49:24.491 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:24.491 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:24.491 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-16T00:49:24.491 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-16T00:49:24.491 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-16T00:49:24.492 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 182, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 443213, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 182, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:49:24.493 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 182, 2024-09-16T00:49:24.494 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 443213, 2024-09-16T00:49:24.494 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:49:24.494 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:24.494 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:24.494 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:24.494 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:24.494 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:24.494 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T00:49:25.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:24 smithi081 ceph-mon[35539]: pgmap v604: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-16T00:49:25.261 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:24 smithi102 ceph-mon[35364]: pgmap v604: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-16T00:49:25.289 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.c 2024-09-16T00:49:25.289 DEBUG:tasks.check_counter:Getting stats from f 2024-09-16T00:49:25.289 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-16T00:49:25.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:24 smithi031 ceph-mon[30956]: pgmap v604: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-16T00:49:25.540 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:49:26.099 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T00:49:26.099 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:26.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 0, 2024-09-16T00:49:26.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2, 2024-09-16T00:49:26.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:49:26.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 0, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.001115554, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000210330, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:26.100 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 182, 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 179, 2024-09-16T00:49:26.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 469072, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 109648, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.025383120, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.011260903, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.018851566, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 178, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.009873811, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000055470 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 469072, 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 109648 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 5, 2024-09-16T00:49:26.103 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 137, 2024-09-16T00:49:26.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 3152, 2024-09-16T00:49:26.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 102688, 2024-09-16T00:49:26.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:26.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:26.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.009743866, 2024-09-16T00:49:26.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.005627934, 2024-09-16T00:49:26.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.002730466, 2024-09-16T00:49:26.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:26.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 136, 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.011212085, 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000082441 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 102688 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.105 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:26.106 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:26.107 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:26.108 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148985, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 30, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:26.109 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 183, 2024-09-16T00:49:26.110 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 443343, 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 183, 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 183, 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 443343, 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:26.111 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T00:49:26.798 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-09-16T00:49:26.799 DEBUG:tasks.check_counter:Getting stats from i 2024-09-16T00:49:26.799 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-16T00:49:27.047 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:49:27.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:26 smithi102 ceph-mon[35364]: pgmap v605: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:27.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:26 smithi081 ceph-mon[35539]: pgmap v605: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:27.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:26 smithi031 ceph-mon[30956]: pgmap v605: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:27.641 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 1523, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 1519, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 151881172, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 533644, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 7, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.218361922, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.047038132, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.126280257, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.038416180, 2024-09-16T00:49:27.642 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1518, 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.064547259, 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000042521 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 484176, 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 110064 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.643 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 734, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 865, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 222706696, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 535984, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 7, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.168177186, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.020980535, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.124213271, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.017143018, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 864, 2024-09-16T00:49:27.644 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.041861941, 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000048451 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 304096 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 639, 2024-09-16T00:49:27.645 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 641, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 163749714, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 203209, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 6, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 5, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.122519792, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.016690432, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.085977850, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.015406685, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 639, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.021221210, 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000033210 2024-09-16T00:49:27.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.647 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 4076, 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.215959647, 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000052983 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.648 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 0, 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 0, 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 0, 2024-09-16T00:49:27.649 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 0, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 38335, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 9562, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 28773, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 24674, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 0, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 0, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 0, 2024-09-16T00:49:27.650 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 2, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2024-09-16T00:49:27.651 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 4, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 234, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:49:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 0 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2024-09-16T00:49:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:49:27.654 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-09-16T00:49:27.655 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 251467, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 133, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 22, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-09-16T00:49:27.656 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 351994489, 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 541263040, 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 541261902, 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 251467 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 24, 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 38334, 2024-09-16T00:49:27.657 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 38310, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 19, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 1198, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 1179, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 38335, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 82174, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 43839, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 0, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 0, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 0, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 128224, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 215328 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.658 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 0, 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 0, 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 0, 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 0, 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 0, 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.659 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-09-16T00:49:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-09-16T00:49:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.661 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.664 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-09-16T00:49:27.664 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.667 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-09-16T00:49:27.667 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.667 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.667 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.667 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-09-16T00:49:27.667 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.669 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.670 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.671 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-09-16T00:49:27.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-09-16T00:49:27.674 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 3, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 2, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 0, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 0, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 1866, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:27.675 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:27.676 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:27.677 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 238684, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 77, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-16T00:49:27.678 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 8880, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 169, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 27066170, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 153726, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-16T00:49:27.679 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 2700, 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 26, 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 4, 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 2700, 2024-09-16T00:49:27.680 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2700, 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 10.002776696, 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003704732 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2700, 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2701 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 2700, 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 2700, 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 0, 2024-09-16T00:49:27.681 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 1677, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 1020, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-09-16T00:49:27.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 2, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:49:27.683 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-09-16T00:49:27.684 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 91, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 4, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 11, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 11, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 0, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 2, 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 0 2024-09-16T00:49:27.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 0, 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:49:27.686 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 0, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 0, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 0, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 2892, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 538064481, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 2892, 2024-09-16T00:49:27.687 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2892, 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 538064481, 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:49:27.688 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 2700, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 537111466, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2700, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 537111466, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.689 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 2700, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 2700, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2700, 2024-09-16T00:49:27.690 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 2700, 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:49:27.691 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x563bc9650b40": { 2024-09-16T00:49:27.692 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:27.693 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:27.694 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:27.694 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:27.694 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T00:49:28.373 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-16T00:49:28.373 DEBUG:tasks.check_counter:Getting stats from l 2024-09-16T00:49:28.374 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-16T00:49:28.617 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:49:29.017 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:28 smithi102 ceph-mon[35364]: pgmap v606: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:49:29.125 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T00:49:29.125 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:49:29.125 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 186, 2024-09-16T00:49:29.125 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 183, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 470160, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 112208, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.027303655, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.011210959, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.020999281, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 182, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.010320287, 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000056704 2024-09-16T00:49:29.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 470160, 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 112208 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 1, 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 139, 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:49:29.127 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 102944, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 3, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 2, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.007971249, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.005750075, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000776003, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 137, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.008620206, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000062921 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:29.128 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 102944 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 4, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 4, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 2992, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 448, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-09-16T00:49:29.129 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.002266451, 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000038930, 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.002235690, 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3, 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.003997532, 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001332510 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:29.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:49:29.131 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-16T00:49:29.132 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-16T00:49:29.133 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-16T00:49:29.134 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:49:29.134 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:49:29.134 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:49:29.134 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:49:29.134 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148984, 2024-09-16T00:49:29.134 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 30, 2024-09-16T00:49:29.134 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:49:29.134 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-16T00:49:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 187, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 443863, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:49:29.136 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 187, 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 187, 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 443863, 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:49:29.137 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T00:49:29.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:28 smithi081 ceph-mon[35539]: pgmap v606: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:49:29.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:28 smithi031 ceph-mon[30956]: pgmap v606: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-16T00:49:29.816 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-16T00:49:29.816 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T00:49:29.827 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T00:49:29.827 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:49:29.827 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:49:29.846 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:49:29.846 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:49:29.904 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:49:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.715+0000 7fde9e85c640 1 -- 172.21.15.31:0/3789909331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 msgr2=0x7fde9810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.715+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3789909331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 0x7fde9810eaa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fde90009ca0 tx=0x7fde9002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.715+0000 7fde9e85c640 1 -- 172.21.15.31:0/3789909331 shutdown_connections 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.715+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3789909331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 0x7fde9810eaa0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.715+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3789909331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde981027a0 0x7fde981067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.715+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3789909331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde98101df0 0x7fde981021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.715+0000 7fde9e85c640 1 -- 172.21.15.31:0/3789909331 >> 172.21.15.31:0/3789909331 conn(0x7fde980fd8b0 msgr2=0x7fde980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.715+0000 7fde9e85c640 1 -- 172.21.15.31:0/3789909331 shutdown_connections 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.715+0000 7fde9e85c640 1 -- 172.21.15.31:0/3789909331 wait complete. 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9e85c640 1 Processor -- start 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9e85c640 1 -- start start 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9e85c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde98101df0 0x7fde981983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9e85c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde981027a0 0x7fde98198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9e85c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 0x7fde9819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:30.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9e85c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fde98111f80 con 0x7fde98106f20 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9e85c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fde98111e00 con 0x7fde981027a0 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9e85c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde98112100 con 0x7fde98101df0 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9cdf9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 0x7fde9819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9cdf9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 0x7fde9819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51556/0 (socket says 172.21.15.31:51556) 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9cdf9640 1 -- 172.21.15.31:0/3133249117 learned_addr learned my addr 172.21.15.31:0/3133249117 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde97fff640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde98101df0 0x7fde981983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde8ffff640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde981027a0 0x7fde98198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9cdf9640 1 -- 172.21.15.31:0/3133249117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde98101df0 msgr2=0x7fde981983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.716+0000 7fde9cdf9640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde98101df0 0x7fde981983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde9cdf9640 1 -- 172.21.15.31:0/3133249117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde981027a0 msgr2=0x7fde98198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde9cdf9640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde981027a0 0x7fde98198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde9cdf9640 1 -- 172.21.15.31:0/3133249117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9819d390 con 0x7fde98106f20 2024-09-16T00:49:30.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde8ffff640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde981027a0 0x7fde98198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde9cdf9640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 0x7fde9819ccb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fde90031a20 tx=0x7fde90031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde95ffb640 1 -- 172.21.15.31:0/3133249117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90003c30 con 0x7fde98106f20 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9819d620 con 0x7fde98106f20 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde95ffb640 1 -- 172.21.15.31:0/3133249117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde90003dd0 con 0x7fde98106f20 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde95ffb640 1 -- 172.21.15.31:0/3133249117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9002fcf0 con 0x7fde98106f20 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.717+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde981a4f60 con 0x7fde98106f20 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.718+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde981021d0 con 0x7fde98106f20 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.719+0000 7fde95ffb640 1 -- 172.21.15.31:0/3133249117 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fde90004050 con 0x7fde98106f20 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.720+0000 7fde95ffb640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde840a3240 0x7fde840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.720+0000 7fde95ffb640 1 -- 172.21.15.31:0/3133249117 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fde900f1010 con 0x7fde98106f20 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.720+0000 7fde97fff640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde840a3240 0x7fde840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.720+0000 7fde97fff640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde840a3240 0x7fde840a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fde8800bbe0 tx=0x7fde8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.724+0000 7fde95ffb640 1 -- 172.21.15.31:0/3133249117 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde900bda10 con 0x7fde98106f20 2024-09-16T00:49:30.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.913+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde9810a100 con 0x7fde98106f20 2024-09-16T00:49:30.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.914+0000 7fde95ffb640 1 -- 172.21.15.31:0/3133249117 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+588 (secure 0 0 0) 0x7fde9002fe90 con 0x7fde98106f20 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2779988710 2024-09-17T00:31:57.569184+0000 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/4219649043 2024-09-17T00:31:57.569184+0000 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/945234876 2024-09-17T00:31:33.279333+0000 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3876592548 2024-09-17T00:31:33.279333+0000 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/4219649043 2024-09-17T00:31:57.569184+0000 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/945234876 2024-09-17T00:31:33.279333+0000 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/876405889 2024-09-17T00:31:33.279333+0000 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2258150314 2024-09-17T00:31:57.569184+0000 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/880818822 2024-09-17T00:31:57.569184+0000 2024-09-16T00:49:30.915 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2222283837 2024-09-17T00:31:33.279333+0000 2024-09-16T00:49:30.916 INFO:teuthology.orchestra.run.smithi031.stderr:listed 10 entries 2024-09-16T00:49:30.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde840a3240 msgr2=0x7fde840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde840a3240 0x7fde840a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fde8800bbe0 tx=0x7fde8800b390 comp rx=0 tx=0).stop 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 msgr2=0x7fde9819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 0x7fde9819ccb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fde90031a20 tx=0x7fde90031a50 comp rx=0 tx=0).stop 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 shutdown_connections 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde98106f20 0x7fde9819ccb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde981027a0 0x7fde98198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde840a3240 0x7fde840a5700 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 --2- 172.21.15.31:0/3133249117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde98101df0 0x7fde981983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 >> 172.21.15.31:0/3133249117 conn(0x7fde980fd8b0 msgr2=0x7fde980febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 shutdown_connections 2024-09-16T00:49:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:30.916+0000 7fde9e85c640 1 -- 172.21.15.31:0/3133249117 wait complete. 2024-09-16T00:49:31.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:30 smithi081 ceph-mon[35539]: pgmap v607: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-16T00:49:31.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:30 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3133249117' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:49:31.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:30 smithi102 ceph-mon[35364]: pgmap v607: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-16T00:49:31.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:30 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3133249117' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:49:31.372 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:30 smithi031 ceph-mon[30956]: pgmap v607: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-16T00:49:31.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:30 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3133249117' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:49:31.549 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T00:49:31.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:49:31.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T00:49:31.595 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:49:31.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:49:31.596 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T00:49:31.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:49:31.629 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T00:49:31.686 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T00:49:31.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:49:31.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:49:31.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:49:31.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-16T00:49:31.860 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T00:49:31.872 INFO:tasks.cephadm:Teardown begin 2024-09-16T00:49:31.872 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T00:49:31.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:49:32.135 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.615+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/3126715760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54103780 msgr2=0x7f6c54107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.615+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/3126715760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54103780 0x7f6c54107bc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009930 tx=0x7f6c4802a420 comp rx=0 tx=0).stop 2024-09-16T00:49:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.615+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/3126715760 shutdown_connections 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.615+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/3126715760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c54108300 0x7f6c5410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.615+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/3126715760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54103780 0x7f6c54107bc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.615+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/3126715760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c54102dd0 0x7f6c541031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.615+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/3126715760 >> 172.21.15.31:0/3126715760 conn(0x7f6c540fc7d0 msgr2=0x7f6c540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.615+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/3126715760 shutdown_connections 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.615+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/3126715760 wait complete. 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5bbe9640 1 Processor -- start 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5bbe9640 1 -- start start 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5bbe9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c54102dd0 0x7f6c541a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5bbe9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c54103780 0x7f6c541a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5bbe9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54108300 0x7f6c5419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5bbe9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54112070 con 0x7f6c54108300 2024-09-16T00:49:32.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5bbe9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54111ef0 con 0x7f6c54102dd0 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5bbe9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541121f0 con 0x7f6c54103780 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5a186640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54108300 0x7f6c5419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5a186640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54108300 0x7f6c5419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51578/0 (socket says 172.21.15.31:51578) 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5a186640 1 -- 172.21.15.31:0/2485504087 learned_addr learned my addr 172.21.15.31:0/2485504087 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c59985640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c54102dd0 0x7f6c541a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5a186640 1 -- 172.21.15.31:0/2485504087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c54103780 msgr2=0x7f6c541a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c59184640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c54103780 0x7f6c541a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5a186640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c54103780 0x7f6c541a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5a186640 1 -- 172.21.15.31:0/2485504087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c54102dd0 msgr2=0x7f6c541a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5a186640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c54102dd0 0x7f6c541a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c5a186640 1 -- 172.21.15.31:0/2485504087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5419ad10 con 0x7f6c54108300 2024-09-16T00:49:32.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.616+0000 7f6c59985640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c54102dd0 0x7f6c541a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.617+0000 7f6c5a186640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54108300 0x7f6c5419a5e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c970 tx=0x7f6c4c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.617+0000 7f6c52ffd640 1 -- 172.21.15.31:0/2485504087 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c007bb0 con 0x7f6c54108300 2024-09-16T00:49:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.617+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5419b000 con 0x7f6c54108300 2024-09-16T00:49:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.617+0000 7f6c52ffd640 1 -- 172.21.15.31:0/2485504087 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4c007d50 con 0x7f6c54108300 2024-09-16T00:49:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.617+0000 7f6c52ffd640 1 -- 172.21.15.31:0/2485504087 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c018800 con 0x7f6c54108300 2024-09-16T00:49:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.617+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c541a72f0 con 0x7f6c54108300 2024-09-16T00:49:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.618+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c541031b0 con 0x7f6c54108300 2024-09-16T00:49:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.619+0000 7f6c52ffd640 1 -- 172.21.15.31:0/2485504087 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6c4c00fa50 con 0x7f6c54108300 2024-09-16T00:49:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.620+0000 7f6c52ffd640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c3c0a3240 0x7f6c3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.620+0000 7f6c59985640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c3c0a3240 0x7f6c3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.620+0000 7f6c52ffd640 1 -- 172.21.15.31:0/2485504087 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f6c4c0cff40 con 0x7f6c54108300 2024-09-16T00:49:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.620+0000 7f6c59985640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c3c0a3240 0x7f6c3c0a5700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6c44005c40 tx=0x7f6c440082f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:32.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.623+0000 7f6c52ffd640 1 -- 172.21.15.31:0/2485504087 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c4c014030 con 0x7f6c54108300 2024-09-16T00:49:32.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.834+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c5419bb10 con 0x7f6c54108300 2024-09-16T00:49:32.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.835+0000 7f6c52ffd640 1 -- 172.21.15.31:0/2485504087 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f6c5419bb10 con 0x7f6c54108300 2024-09-16T00:49:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:49:32.835 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:49:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:49:32.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c3c0a3240 msgr2=0x7f6c3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:32.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c3c0a3240 0x7f6c3c0a5700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6c44005c40 tx=0x7f6c440082f0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54108300 msgr2=0x7f6c5419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54108300 0x7f6c5419a5e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c970 tx=0x7f6c4c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:49:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 shutdown_connections 2024-09-16T00:49:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54108300 0x7f6c5419a5e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f6c3c0a3240 0x7f6c3c0a5700 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c54103780 0x7f6c541a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 --2- 172.21.15.31:0/2485504087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c54102dd0 0x7f6c541a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 >> 172.21.15.31:0/2485504087 conn(0x7f6c540fc7d0 msgr2=0x7f6c54100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 shutdown_connections 2024-09-16T00:49:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:32.837+0000 7f6c5bbe9640 1 -- 172.21.15.31:0/2485504087 wait complete. 2024-09-16T00:49:33.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:32 smithi031 ceph-mon[30956]: pgmap v608: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:33.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:32 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/2485504087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:49:33.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:32 smithi081 ceph-mon[35539]: pgmap v608: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:33.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:32 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/2485504087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:49:33.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:32 smithi102 ceph-mon[35364]: pgmap v608: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:49:33.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:32 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/2485504087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:49:33.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:49:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.273+0000 7f9fd6d65640 1 -- 172.21.15.31:0/1436066699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 msgr2=0x7f9fd010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.273+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/1436066699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 0x7f9fd010e600 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009ca0 tx=0x7f9fc802a950 comp rx=0 tx=0).stop 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.273+0000 7f9fd6d65640 1 -- 172.21.15.31:0/1436066699 shutdown_connections 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.273+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/1436066699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 0x7f9fd010e600 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.273+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/1436066699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01036c0 0x7f9fd010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.273+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/1436066699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fd0102da0 0x7f9fd0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.273+0000 7f9fd6d65640 1 -- 172.21.15.31:0/1436066699 >> 172.21.15.31:0/1436066699 conn(0x7f9fd00fc7a0 msgr2=0x7f9fd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.273+0000 7f9fd6d65640 1 -- 172.21.15.31:0/1436066699 shutdown_connections 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.273+0000 7f9fd6d65640 1 -- 172.21.15.31:0/1436066699 wait complete. 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd6d65640 1 Processor -- start 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd6d65640 1 -- start start 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd6d65640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fd0102da0 0x7f9fd01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd6d65640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01036c0 0x7f9fd01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd6d65640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 0x7f9fd019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd6d65640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd0111f60 con 0x7f9fd010c180 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd6d65640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd0111de0 con 0x7f9fd0102da0 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd6d65640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01120e0 con 0x7f9fd01036c0 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd5302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 0x7f9fd019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd5302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 0x7f9fd019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51606/0 (socket says 172.21.15.31:51606) 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd5302640 1 -- 172.21.15.31:0/505516751 learned_addr learned my addr 172.21.15.31:0/505516751 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd4b01640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fd0102da0 0x7f9fd01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd5302640 1 -- 172.21.15.31:0/505516751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01036c0 msgr2=0x7f9fd01988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd5302640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01036c0 0x7f9fd01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd5302640 1 -- 172.21.15.31:0/505516751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fd0102da0 msgr2=0x7f9fd01983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd5302640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fd0102da0 0x7f9fd01983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd5302640 1 -- 172.21.15.31:0/505516751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd019d350 con 0x7f9fd010c180 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.274+0000 7f9fd4b01640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fd0102da0 0x7f9fd01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:34.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.275+0000 7f9fd5302640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 0x7f9fd019cc70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9fc802ae70 tx=0x7f9fc802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.275+0000 7f9fcdffb640 1 -- 172.21.15.31:0/505516751 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8034040 con 0x7f9fd010c180 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.275+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd019d5e0 con 0x7f9fd010c180 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.275+0000 7f9fcdffb640 1 -- 172.21.15.31:0/505516751 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc8030920 con 0x7f9fd010c180 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.275+0000 7f9fcdffb640 1 -- 172.21.15.31:0/505516751 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8045480 con 0x7f9fd010c180 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.275+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd01a4f20 con 0x7f9fd010c180 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.276+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd006b010 con 0x7f9fd010c180 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.277+0000 7f9fcdffb640 1 -- 172.21.15.31:0/505516751 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9fc8037020 con 0x7f9fd010c180 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.278+0000 7f9fcdffb640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fbc0a3240 0x7f9fbc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.278+0000 7f9fd4b01640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fbc0a3240 0x7f9fbc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.278+0000 7f9fcdffb640 1 -- 172.21.15.31:0/505516751 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f9fc80f0cb0 con 0x7f9fd010c180 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.278+0000 7f9fd4b01640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fbc0a3240 0x7f9fbc0a5700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80058c0 tx=0x7f9fb8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.282+0000 7f9fcdffb640 1 -- 172.21.15.31:0/505516751 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fc80bd620 con 0x7f9fd010c180 2024-09-16T00:49:34.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.492+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fd01a5120 con 0x7f9fd010c180 2024-09-16T00:49:34.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.493+0000 7f9fcdffb640 1 -- 172.21.15.31:0/505516751 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11848 (secure 0 0 0) 0x7f9fc80bcf20 con 0x7f9fd010c180 2024-09-16T00:49:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:49:34.570 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T00:39:36:131244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3753760611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3753760611},{"type":"v1","addr":"172.21.15.31:6835","nonce":3753760611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3116907098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3116907098},{"type":"v1","addr":"172.21.15.31:6837","nonce":3116907098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3406901465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3406901465},{"type":"v1","addr":"172.21.15.31:6839","nonce":3406901465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3311366678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3311366678},{"type":"v1","addr":"172.21.15.31:6841","nonce":3311366678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1467259017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1467259017},{"type":"v1","addr":"172.21.15.81:6833","nonce":1467259017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/316287545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":316287545},{"type":"v1","addr":"172.21.15.102:6833","nonce":316287545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1522041920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1522041920},{"type":"v1","addr":"172.21.15.81:6835","nonce":1522041920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1338256650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1338256650},{"type":"v1","addr":"172.21.15.81:6837","nonce":1338256650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1825124874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1825124874},{"type":"v1","addr":"172.21.15.102:6835","nonce":1825124874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/272579536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":272579536},{"type":"v1","addr":"172.21.15.102:6837","nonce":272579536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:38:53.576781+0000","modified":"2024-09-16T00:39:35.954166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6839/3021718993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3021718993},{"type":"v1","addr":"172.21.15.102:6839","nonce":3021718993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/2243721734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2243721734},{"type":"v1","addr":"172.21.15.81:6839","nonce":2243721734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-09-16T00:49:34.571 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T00:49:34.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fbc0a3240 msgr2=0x7f9fbc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:34.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fbc0a3240 0x7f9fbc0a5700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80058c0 tx=0x7f9fb8005850 comp rx=0 tx=0).stop 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 msgr2=0x7f9fd019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 0x7f9fd019cc70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9fc802ae70 tx=0x7f9fc802f5b0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 shutdown_connections 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fd010c180 0x7f9fd019cc70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f9fbc0a3240 0x7f9fbc0a5700 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01036c0 0x7f9fd01988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 --2- 172.21.15.31:0/505516751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fd0102da0 0x7f9fd01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 >> 172.21.15.31:0/505516751 conn(0x7f9fd00fc7a0 msgr2=0x7f9fd00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 shutdown_connections 2024-09-16T00:49:34.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:34.495+0000 7f9fd6d65640 1 -- 172.21.15.31:0/505516751 wait complete. 2024-09-16T00:49:35.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:34 smithi031 ceph-mon[30956]: pgmap v609: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-09-16T00:49:35.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:34 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/505516751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:49:35.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:49:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:34 smithi081 ceph-mon[35539]: pgmap v609: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-09-16T00:49:35.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:34 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/505516751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:49:35.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:34 smithi102 ceph-mon[35364]: pgmap v609: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-09-16T00:49:35.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:34 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/505516751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:49:35.454 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.982+0000 7f646b40e640 1 -- 172.21.15.31:0/364293371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 msgr2=0x7f6464106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.982+0000 7f646b40e640 1 --2- 172.21.15.31:0/364293371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 0x7f6464106e60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6460009930 tx=0x7f646002a420 comp rx=0 tx=0).stop 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.982+0000 7f646b40e640 1 -- 172.21.15.31:0/364293371 shutdown_connections 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.982+0000 7f646b40e640 1 --2- 172.21.15.31:0/364293371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464101420 0x7f646410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.982+0000 7f646b40e640 1 --2- 172.21.15.31:0/364293371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6464100a80 0x7f6464100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.982+0000 7f646b40e640 1 --2- 172.21.15.31:0/364293371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 0x7f6464106e60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.982+0000 7f646b40e640 1 -- 172.21.15.31:0/364293371 >> 172.21.15.31:0/364293371 conn(0x7f64640fc7a0 msgr2=0x7f64640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.982+0000 7f646b40e640 1 -- 172.21.15.31:0/364293371 shutdown_connections 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.982+0000 7f646b40e640 1 -- 172.21.15.31:0/364293371 wait complete. 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f646b40e640 1 Processor -- start 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f646b40e640 1 -- start start 2024-09-16T00:49:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f646b40e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464100a80 0x7f6464102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f646b40e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6464101420 0x7f6464102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f646b40e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 0x7f64641030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f646b40e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6464110f20 con 0x7f6464106a80 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f646b40e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6464110da0 con 0x7f6464101420 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f646b40e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64641110a0 con 0x7f6464100a80 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f64699ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 0x7f64641030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f64699ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 0x7f64641030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51628/0 (socket says 172.21.15.31:51628) 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f64699ab640 1 -- 172.21.15.31:0/407899849 learned_addr learned my addr 172.21.15.31:0/407899849 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f64689a9640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6464101420 0x7f6464102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f64691aa640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464100a80 0x7f6464102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f64699ab640 1 -- 172.21.15.31:0/407899849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464100a80 msgr2=0x7f6464102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f64699ab640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464100a80 0x7f6464102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f64699ab640 1 -- 172.21.15.31:0/407899849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6464101420 msgr2=0x7f6464102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.983+0000 7f64699ab640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6464101420 0x7f6464102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.984+0000 7f64699ab640 1 -- 172.21.15.31:0/407899849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64641a6a80 con 0x7f6464106a80 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.984+0000 7f64689a9640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6464101420 0x7f6464102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.984+0000 7f64699ab640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 0x7f64641030d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f645c00b970 tx=0x7f645c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.984+0000 7f645a7fc640 1 -- 172.21.15.31:0/407899849 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c019070 con 0x7f6464106a80 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.984+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64641a6d70 con 0x7f6464106a80 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.984+0000 7f645a7fc640 1 -- 172.21.15.31:0/407899849 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645c00cb40 con 0x7f6464106a80 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.984+0000 7f645a7fc640 1 -- 172.21.15.31:0/407899849 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c007650 con 0x7f6464106a80 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.984+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64641a7260 con 0x7f6464106a80 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.985+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6464100f60 con 0x7f6464106a80 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.986+0000 7f645a7fc640 1 -- 172.21.15.31:0/407899849 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f645c01c070 con 0x7f6464106a80 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.987+0000 7f645a7fc640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64500a3240 0x7f64500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.987+0000 7f645a7fc640 1 -- 172.21.15.31:0/407899849 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7f645c0cf390 con 0x7f6464106a80 2024-09-16T00:49:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.987+0000 7f64691aa640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64500a3240 0x7f64500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.988+0000 7f64691aa640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64500a3240 0x7f64500a5700 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6460000c00 tx=0x7f6460031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:35.992+0000 7f645a7fc640 1 -- 172.21.15.31:0/407899849 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f645c09bca0 con 0x7f6464106a80 2024-09-16T00:49:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.168+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64641a74f0 con 0x7f6464106a80 2024-09-16T00:49:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.170+0000 7f645a7fc640 1 -- 172.21.15.31:0/407899849 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24766 (secure 0 0 0) 0x7f645c09b5a0 con 0x7f6464106a80 2024-09-16T00:49:36.171 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:49:36.171 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"ccb601ea-73c2-11ef-bceb-c7b262605968","created":"2024-09-16T00:31:04.619511+0000","modified":"2024-09-16T00:40:59.575845+0000","last_up_change":"2024-09-16T00:38:01.215601+0000","last_in_change":"2024-09-16T00:37:36.419355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:34:12.067708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:38:49.806646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:38:51.945160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:38:58.457307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d076c77e-9b90-46b1-8e52-eecb4d02d169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6803","nonce":1435627868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6805","nonce":1435627868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6809","nonce":1435627868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1435627868},{"type":"v1","addr":"172.21.15.31:6807","nonce":1435627868}]},"public_addr":"172.21.15.31:6803/1435627868","cluster_addr":"172.21.15.31:6805/1435627868","heartbeat_back_addr":"172.21.15.31:6809/1435627868","heartbeat_front_addr":"172.21.15.31:6807/1435627868","state":["exists","up"]},{"osd":1,"uuid":"cc498a19-f3ac-4145-a121-81e710cd1558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6801","nonce":3628874088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6803","nonce":3628874088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6807","nonce":3628874088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3628874088},{"type":"v1","addr":"172.21.15.81:6805","nonce":3628874088}]},"public_addr":"172.21.15.81:6801/3628874088","cluster_addr":"172.21.15.81:6803/3628874088","heartbeat_back_addr":"172.21.15.81:6807/3628874088","heartbeat_front_addr":"172.21.15.81:6805/3628874088","state":["exists","up"]},{"osd":2,"uuid":"761075fe-1766-4924-afa3-6e2b1b3c3a37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6801","nonce":2755894572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6803","nonce":2755894572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6807","nonce":2755894572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2755894572},{"type":"v1","addr":"172.21.15.102:6805","nonce":2755894572}]},"public_addr":"172.21.15.102:6801/2755894572","cluster_addr":"172.21.15.102:6803/2755894572","heartbeat_back_addr":"172.21.15.102:6807/2755894572","heartbeat_front_addr":"172.21.15.102:6805/2755894572","state":["exists","up"]},{"osd":3,"uuid":"1f362961-16d5-4892-99df-3bec7a940e68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6811","nonce":3958016219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6813","nonce":3958016219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6817","nonce":3958016219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3958016219},{"type":"v1","addr":"172.21.15.31:6815","nonce":3958016219}]},"public_addr":"172.21.15.31:6811/3958016219","cluster_addr":"172.21.15.31:6813/3958016219","heartbeat_back_addr":"172.21.15.31:6817/3958016219","heartbeat_front_addr":"172.21.15.31:6815/3958016219","state":["exists","up"]},{"osd":4,"uuid":"2942a206-0e2f-4a98-99d3-efc5325578f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6809","nonce":59224774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6811","nonce":59224774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6815","nonce":59224774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":59224774},{"type":"v1","addr":"172.21.15.81:6813","nonce":59224774}]},"public_addr":"172.21.15.81:6809/59224774","cluster_addr":"172.21.15.81:6811/59224774","heartbeat_back_addr":"172.21.15.81:6815/59224774","heartbeat_front_addr":"172.21.15.81:6813/59224774","state":["exists","up"]},{"osd":5,"uuid":"893456fd-1776-4a46-b16c-4590c38b3ee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6809","nonce":215828552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6811","nonce":215828552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6815","nonce":215828552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":215828552},{"type":"v1","addr":"172.21.15.102:6813","nonce":215828552}]},"public_addr":"172.21.15.102:6809/215828552","cluster_addr":"172.21.15.102:6811/215828552","heartbeat_back_addr":"172.21.15.102:6815/215828552","heartbeat_front_addr":"172.21.15.102:6813/215828552","state":["exists","up"]},{"osd":6,"uuid":"749c866d-0f7f-4a78-ac75-5728ed7d49ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6819","nonce":1424825752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6821","nonce":1424825752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6825","nonce":1424825752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1424825752},{"type":"v1","addr":"172.21.15.31:6823","nonce":1424825752}]},"public_addr":"172.21.15.31:6819/1424825752","cluster_addr":"172.21.15.31:6821/1424825752","heartbeat_back_addr":"172.21.15.31:6825/1424825752","heartbeat_front_addr":"172.21.15.31:6823/1424825752","state":["exists","up"]},{"osd":7,"uuid":"3f7eade8-dd2f-42ce-8b27-0c03f9172b79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6817","nonce":269260316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6819","nonce":269260316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6823","nonce":269260316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":269260316},{"type":"v1","addr":"172.21.15.81:6821","nonce":269260316}]},"public_addr":"172.21.15.81:6817/269260316","cluster_addr":"172.21.15.81:6819/269260316","heartbeat_back_addr":"172.21.15.81:6823/269260316","heartbeat_front_addr":"172.21.15.81:6821/269260316","state":["exists","up"]},{"osd":8,"uuid":"b2673057-c859-4b46-92af-c376f007503a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6817","nonce":304252744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6819","nonce":304252744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6823","nonce":304252744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":304252744},{"type":"v1","addr":"172.21.15.102:6821","nonce":304252744}]},"public_addr":"172.21.15.102:6817/304252744","cluster_addr":"172.21.15.102:6819/304252744","heartbeat_back_addr":"172.21.15.102:6823/304252744","heartbeat_front_addr":"172.21.15.102:6821/304252744","state":["exists","up"]},{"osd":9,"uuid":"cf12fcdc-621d-44c2-8863-e995d82fb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6827","nonce":3465344894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6829","nonce":3465344894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6833","nonce":3465344894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3465344894},{"type":"v1","addr":"172.21.15.31:6831","nonce":3465344894}]},"public_addr":"172.21.15.31:6827/3465344894","cluster_addr":"172.21.15.31:6829/3465344894","heartbeat_back_addr":"172.21.15.31:6833/3465344894","heartbeat_front_addr":"172.21.15.31:6831/3465344894","state":["exists","up"]},{"osd":10,"uuid":"d229e783-ab63-44ce-816e-bf0a1949c5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6825","nonce":1666424173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6827","nonce":1666424173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6831","nonce":1666424173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":1666424173},{"type":"v1","addr":"172.21.15.81:6829","nonce":1666424173}]},"public_addr":"172.21.15.81:6825/1666424173","cluster_addr":"172.21.15.81:6827/1666424173","heartbeat_back_addr":"172.21.15.81:6831/1666424173","heartbeat_front_addr":"172.21.15.81:6829/1666424173","state":["exists","up"]},{"osd":11,"uuid":"1fddf945-ad39-4a5c-a40b-8ac262a7df75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6825","nonce":1575052259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6827","nonce":1575052259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6831","nonce":1575052259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1575052259},{"type":"v1","addr":"172.21.15.102:6829","nonce":1575052259}]},"public_addr":"172.21.15.102:6825/1575052259","cluster_addr":"172.21.15.102:6827/1575052259","heartbeat_back_addr":"172.21.15.102:6831/1575052259","heartbeat_front_addr":"172.21.15.102:6829/1575052259","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:18.489920+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:33:43.675223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:08.130970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:34:34.971443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:00.051071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:26.033230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:35:50.453382+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:16.109643+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:36:41.026613+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:07.007680+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:32.394484+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:37:58.553953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2779988710":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6801/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/3876592548":"2024-09-17T00:31:33.279333+0000","172.21.15.31:6801/4219649043":"2024-09-17T00:31:57.569184+0000","172.21.15.31:6800/945234876":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/876405889":"2024-09-17T00:31:33.279333+0000","172.21.15.31:0/2258150314":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/880818822":"2024-09-17T00:31:57.569184+0000","172.21.15.31:0/2222283837":"2024-09-17T00:31:33.279333+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64500a3240 msgr2=0x7f64500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64500a3240 0x7f64500a5700 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6460000c00 tx=0x7f6460031490 comp rx=0 tx=0).stop 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 msgr2=0x7f64641030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 0x7f64641030d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f645c00b970 tx=0x7f645c00be30 comp rx=0 tx=0).stop 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 shutdown_connections 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6464106a80 0x7f64641030d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6464101420 0x7f6464102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f64500a3240 0x7f64500a5700 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 --2- 172.21.15.31:0/407899849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464100a80 0x7f6464102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 >> 172.21.15.31:0/407899849 conn(0x7f64640fc7a0 msgr2=0x7f64641048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:36.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 shutdown_connections 2024-09-16T00:49:36.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:36.172+0000 7f646b40e640 1 -- 172.21.15.31:0/407899849 wait complete. 2024-09-16T00:49:36.981 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-16T00:49:37.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:36 smithi081 ceph-mon[35539]: pgmap v610: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:37.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:36 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/407899849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:49:37.248 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:37.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:36 smithi102 ceph-mon[35364]: pgmap v610: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:37.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:36 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/407899849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:49:37.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:36 smithi031 ceph-mon[30956]: pgmap v610: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:37.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:36 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/407899849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:49:37.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.713+0000 7fde4ff82640 1 -- 172.21.15.31:0/1331651552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48102790 msgr2=0x7fde481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.713+0000 7fde4ff82640 1 --2- 172.21.15.31:0/1331651552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48102790 0x7fde481067e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009900 tx=0x7fde3c02a410 comp rx=0 tx=0).stop 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.714+0000 7fde4ff82640 1 -- 172.21.15.31:0/1331651552 shutdown_connections 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.714+0000 7fde4ff82640 1 --2- 172.21.15.31:0/1331651552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde48106f10 0x7fde4810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.714+0000 7fde4ff82640 1 --2- 172.21.15.31:0/1331651552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48102790 0x7fde481067e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.714+0000 7fde4ff82640 1 --2- 172.21.15.31:0/1331651552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde48101de0 0x7fde481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.714+0000 7fde4ff82640 1 -- 172.21.15.31:0/1331651552 >> 172.21.15.31:0/1331651552 conn(0x7fde480fd8c0 msgr2=0x7fde480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.714+0000 7fde4ff82640 1 -- 172.21.15.31:0/1331651552 shutdown_connections 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.714+0000 7fde4ff82640 1 -- 172.21.15.31:0/1331651552 wait complete. 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.714+0000 7fde4ff82640 1 Processor -- start 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.714+0000 7fde4ff82640 1 -- start start 2024-09-16T00:49:37.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4ff82640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48101de0 0x7fde48198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4ff82640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde48102790 0x7fde48198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4ff82640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde48106f10 0x7fde4819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4ff82640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fde48111f70 con 0x7fde48101de0 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4ff82640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fde48111df0 con 0x7fde48106f10 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4ff82640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde481120f0 con 0x7fde48102790 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4dd1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48101de0 0x7fde48198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4dd1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48101de0 0x7fde48198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35810/0 (socket says 172.21.15.31:35810) 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4dd1e640 1 -- 172.21.15.31:0/3198173097 learned_addr learned my addr 172.21.15.31:0/3198173097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4d51d640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde48102790 0x7fde48198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4e51f640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde48106f10 0x7fde4819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4dd1e640 1 -- 172.21.15.31:0/3198173097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde48102790 msgr2=0x7fde48198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:37.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4dd1e640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde48102790 0x7fde48198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4dd1e640 1 -- 172.21.15.31:0/3198173097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde48106f10 msgr2=0x7fde4819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4dd1e640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde48106f10 0x7fde4819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4dd1e640 1 -- 172.21.15.31:0/3198173097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4819d3e0 con 0x7fde48101de0 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4e51f640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde48106f10 0x7fde4819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.715+0000 7fde4dd1e640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48101de0 0x7fde48198430 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d950 tx=0x7fde4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.716+0000 7fde46ffd640 1 -- 172.21.15.31:0/3198173097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40017070 con 0x7fde48101de0 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.716+0000 7fde46ffd640 1 -- 172.21.15.31:0/3198173097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde40009c30 con 0x7fde48101de0 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.716+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4819d6d0 con 0x7fde48101de0 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.716+0000 7fde46ffd640 1 -- 172.21.15.31:0/3198173097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4000f6b0 con 0x7fde48101de0 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.716+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde481a4fb0 con 0x7fde48101de0 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.717+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde481021c0 con 0x7fde48101de0 2024-09-16T00:49:37.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.718+0000 7fde46ffd640 1 -- 172.21.15.31:0/3198173097 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fde400109d0 con 0x7fde48101de0 2024-09-16T00:49:37.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.719+0000 7fde46ffd640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde300a3240 0x7fde300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:37.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.719+0000 7fde4d51d640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde300a3240 0x7fde300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:37.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.719+0000 7fde46ffd640 1 -- 172.21.15.31:0/3198173097 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9174+0+0 (secure 0 0 0) 0x7fde400cf940 con 0x7fde48101de0 2024-09-16T00:49:37.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.719+0000 7fde4d51d640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde300a3240 0x7fde300a5700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009230 tx=0x7fde3c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:37.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.722+0000 7fde46ffd640 1 -- 172.21.15.31:0/3198173097 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde40009de0 con 0x7fde48101de0 2024-09-16T00:49:37.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.933+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fde4810a0f0 con 0x7fde48101de0 2024-09-16T00:49:37.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.936+0000 7fde46ffd640 1 -- 172.21.15.31:0/3198173097 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7fde4009c250 con 0x7fde48101de0 2024-09-16T00:49:37.936 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T00:49:37.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde300a3240 msgr2=0x7fde300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:37.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde300a3240 0x7fde300a5700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009230 tx=0x7fde3c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48101de0 msgr2=0x7fde48198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48101de0 0x7fde48198430 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d950 tx=0x7fde4000de10 comp rx=0 tx=0).stop 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 shutdown_connections 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde48106f10 0x7fde4819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fde300a3240 0x7fde300a5700 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde48102790 0x7fde48198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 --2- 172.21.15.31:0/3198173097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde48101de0 0x7fde48198430 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 >> 172.21.15.31:0/3198173097 conn(0x7fde480fd8c0 msgr2=0x7fde480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 shutdown_connections 2024-09-16T00:49:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:37.939+0000 7fde4ff82640 1 -- 172.21.15.31:0/3198173097 wait complete. 2024-09-16T00:49:38.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3198173097' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:49:38.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-mon[35539]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:49:38.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-mon[35539]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:49:38.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3198173097' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:49:38.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-mon[35539]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:49:38.247 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-mon[35539]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-16T00:49:38.248 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b[65567]: ignoring --setuser ceph since I am not root 2024-09-16T00:49:38.248 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b[65567]: ignoring --setgroup ceph since I am not root 2024-09-16T00:49:38.248 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-mds[65571]: -- 172.21.15.81:0/61019338 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b30a49860 con 0x555b30a27400 2024-09-16T00:49:38.248 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:49:37 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b[65567]: starting mds.b at 2024-09-16T00:49:38.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3198173097' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:49:38.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-mon[35364]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:49:38.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-mon[35364]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:49:38.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3198173097' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:49:38.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-mon[35364]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:49:38.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-mon[35364]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-16T00:49:38.300 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i[65307]: ignoring --setuser ceph since I am not root 2024-09-16T00:49:38.300 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i[65307]: ignoring --setgroup ceph since I am not root 2024-09-16T00:49:38.300 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-mds[65311]: -- 172.21.15.102:0/1775388963 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55efb4c6d860 con 0x55efb4c4a800 2024-09-16T00:49:38.300 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:49:37 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i[65307]: starting mds.i at 2024-09-16T00:49:38.311 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a[30952]: 2024-09-16T00:49:37.934+0000 7fcf94616640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:49:38.311 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:37 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3198173097' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:49:38.311 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:37 smithi031 ceph-mon[30956]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:49:38.311 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:37 smithi031 ceph-mon[30956]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:49:38.311 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:37 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3198173097' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:49:38.311 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:37 smithi031 ceph-mon[30956]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:49:38.311 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:37 smithi031 ceph-mon[30956]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-16T00:49:38.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T00:49:38.920 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:39.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:39 smithi102 ceph-mon[35364]: pgmap v611: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:39.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:39 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.102:6838/1783799602,v1:172.21.15.102:6839/1783799602] up:boot 2024-09-16T00:49:39.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:39 smithi102 ceph-mon[35364]: mds.? [v2:172.21.15.81:6838/3852612118,v1:172.21.15.81:6839/3852612118] up:boot 2024-09-16T00:49:39.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:39 smithi102 ceph-mon[35364]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-16T00:49:39.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:39 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:49:39.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:39 smithi102 ceph-mon[35364]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:49:39.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:39 smithi031 ceph-mon[30956]: pgmap v611: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:39.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:39 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.102:6838/1783799602,v1:172.21.15.102:6839/1783799602] up:boot 2024-09-16T00:49:39.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:39 smithi031 ceph-mon[30956]: mds.? [v2:172.21.15.81:6838/3852612118,v1:172.21.15.81:6839/3852612118] up:boot 2024-09-16T00:49:39.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:39 smithi031 ceph-mon[30956]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-16T00:49:39.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:39 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:49:39.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:39 smithi031 ceph-mon[30956]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:49:39.338 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:39 smithi081 ceph-mon[35539]: pgmap v611: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-16T00:49:39.338 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:39 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.102:6838/1783799602,v1:172.21.15.102:6839/1783799602] up:boot 2024-09-16T00:49:39.338 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:39 smithi081 ceph-mon[35539]: mds.? [v2:172.21.15.81:6838/3852612118,v1:172.21.15.81:6839/3852612118] up:boot 2024-09-16T00:49:39.338 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:39 smithi081 ceph-mon[35539]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-16T00:49:39.338 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:49:39.338 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:39 smithi081 ceph-mon[35539]: from='mgr.14150 172.21.15.31:0/221280233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:49:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.391+0000 7f7b944b4640 1 -- 172.21.15.31:0/3506005194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 msgr2=0x7f7b8c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.391+0000 7f7b944b4640 1 --2- 172.21.15.31:0/3506005194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 0x7f7b8c10d950 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009cd0 tx=0x7f7b8402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:49:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.391+0000 7f7b944b4640 1 -- 172.21.15.31:0/3506005194 shutdown_connections 2024-09-16T00:49:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.391+0000 7f7b944b4640 1 --2- 172.21.15.31:0/3506005194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 0x7f7b8c10d950 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.391+0000 7f7b944b4640 1 --2- 172.21.15.31:0/3506005194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c100a80 0x7f7b8c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.391+0000 7f7b944b4640 1 --2- 172.21.15.31:0/3506005194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b8c106a80 0x7f7b8c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.391+0000 7f7b944b4640 1 -- 172.21.15.31:0/3506005194 >> 172.21.15.31:0/3506005194 conn(0x7f7b8c0fc7a0 msgr2=0x7f7b8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.391+0000 7f7b944b4640 1 -- 172.21.15.31:0/3506005194 shutdown_connections 2024-09-16T00:49:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.391+0000 7f7b944b4640 1 -- 172.21.15.31:0/3506005194 wait complete. 2024-09-16T00:49:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b944b4640 1 Processor -- start 2024-09-16T00:49:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b944b4640 1 -- start start 2024-09-16T00:49:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b944b4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b8c100a80 0x7f7b8c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b944b4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 0x7f7b8c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b944b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c106a80 0x7f7b8c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b944b4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c110e80 con 0x7f7b8c101420 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b944b4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c110d00 con 0x7f7b8c106a80 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b944b4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c111000 con 0x7f7b8c100a80 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b91a4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 0x7f7b8c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b91a4f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 0x7f7b8c1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35836/0 (socket says 172.21.15.31:35836) 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b91a4f640 1 -- 172.21.15.31:0/4119241796 learned_addr learned my addr 172.21.15.31:0/4119241796 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b92250640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b8c100a80 0x7f7b8c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.392+0000 7f7b92a51640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c106a80 0x7f7b8c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b91a4f640 1 -- 172.21.15.31:0/4119241796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b8c100a80 msgr2=0x7f7b8c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b91a4f640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b8c100a80 0x7f7b8c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b91a4f640 1 -- 172.21.15.31:0/4119241796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c106a80 msgr2=0x7f7b8c1a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b91a4f640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c106a80 0x7f7b8c1a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b91a4f640 1 -- 172.21.15.31:0/4119241796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8c1a4850 con 0x7f7b8c101420 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b91a4f640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 0x7f7b8c1997d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00d950 tx=0x7f7b7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b837fe640 1 -- 172.21.15.31:0/4119241796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c017070 con 0x7f7b8c101420 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8c1a4a20 con 0x7f7b8c101420 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b837fe640 1 -- 172.21.15.31:0/4119241796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7c009c30 con 0x7f7b8c101420 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b837fe640 1 -- 172.21.15.31:0/4119241796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00f6b0 con 0x7f7b8c101420 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.393+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8c1a4f30 con 0x7f7b8c101420 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.394+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8c100ee0 con 0x7f7b8c101420 2024-09-16T00:49:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.395+0000 7f7b837fe640 1 -- 172.21.15.31:0/4119241796 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7b7c0109d0 con 0x7f7b8c101420 2024-09-16T00:49:39.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.396+0000 7f7b837fe640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b780a3240 0x7f7b780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:39.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.396+0000 7f7b92250640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b780a3240 0x7f7b780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:39.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.396+0000 7f7b837fe640 1 -- 172.21.15.31:0/4119241796 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9260+0+0 (secure 0 0 0) 0x7f7b7c0cf950 con 0x7f7b8c101420 2024-09-16T00:49:39.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.396+0000 7f7b92250640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b780a3240 0x7f7b780a5700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:39.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.400+0000 7f7b837fe640 1 -- 172.21.15.31:0/4119241796 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b7c009de0 con 0x7f7b8c101420 2024-09-16T00:49:39.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:39.610+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7b8c108ff0 con 0x7f7b8c101420 2024-09-16T00:49:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.014+0000 7f7b837fe640 1 -- 172.21.15.31:0/4119241796 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f7b7c09bbc0 con 0x7f7b8c101420 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b780a3240 msgr2=0x7f7b780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b780a3240 0x7f7b780a5700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b8800b390 comp rx=0 tx=0).stop 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 msgr2=0x7f7b8c1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 0x7f7b8c1997d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00d950 tx=0x7f7b7c00de10 comp rx=0 tx=0).stop 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 shutdown_connections 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c106a80 0x7f7b8c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c101420 0x7f7b8c1997d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f7b780a3240 0x7f7b780a5700 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 --2- 172.21.15.31:0/4119241796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b8c100a80 0x7f7b8c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 >> 172.21.15.31:0/4119241796 conn(0x7f7b8c0fc7a0 msgr2=0x7f7b8c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 shutdown_connections 2024-09-16T00:49:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:40.017+0000 7f7b944b4640 1 -- 172.21.15.31:0/4119241796 wait complete. 2024-09-16T00:49:40.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:40 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4119241796' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:49:40.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:40 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4119241796' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:49:40.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:40 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4119241796' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:49:40.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T00:49:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:41.278 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:41 smithi031 ceph-mon[30956]: pgmap v613: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-16T00:49:41.278 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:41 smithi031 ceph-mon[30956]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:49:41.278 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:41 smithi031 ceph-mon[30956]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:49:41.278 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:41 smithi031 ceph-mon[30956]: Cluster is now healthy 2024-09-16T00:49:41.278 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:41 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/4119241796' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:49:41.278 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:41 smithi031 ceph-mon[30956]: fsmap 12 up:standby 2024-09-16T00:49:41.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:41 smithi102 ceph-mon[35364]: pgmap v613: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-16T00:49:41.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:41 smithi102 ceph-mon[35364]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:49:41.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:41 smithi102 ceph-mon[35364]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:49:41.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:41 smithi102 ceph-mon[35364]: Cluster is now healthy 2024-09-16T00:49:41.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:41 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/4119241796' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:49:41.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:41 smithi102 ceph-mon[35364]: fsmap 12 up:standby 2024-09-16T00:49:41.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.460+0000 7f4d2db81640 1 -- 172.21.15.31:0/1184987770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28103660 msgr2=0x7f4d2810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:41.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.460+0000 7f4d2db81640 1 --2- 172.21.15.31:0/1184987770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28103660 0x7f4d2810bb50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009900 tx=0x7f4d1c02a480 comp rx=0 tx=0).stop 2024-09-16T00:49:41.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.460+0000 7f4d2db81640 1 -- 172.21.15.31:0/1184987770 shutdown_connections 2024-09-16T00:49:41.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.460+0000 7f4d2db81640 1 --2- 172.21.15.31:0/1184987770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2810c170 0x7f4d2810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:41.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.460+0000 7f4d2db81640 1 --2- 172.21.15.31:0/1184987770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28103660 0x7f4d2810bb50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:41.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.460+0000 7f4d2db81640 1 --2- 172.21.15.31:0/1184987770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d28102d40 0x7f4d28103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:41.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.460+0000 7f4d2db81640 1 -- 172.21.15.31:0/1184987770 >> 172.21.15.31:0/1184987770 conn(0x7f4d280fc7a0 msgr2=0x7f4d280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.460+0000 7f4d2db81640 1 -- 172.21.15.31:0/1184987770 shutdown_connections 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.461+0000 7f4d2db81640 1 -- 172.21.15.31:0/1184987770 wait complete. 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.461+0000 7f4d2db81640 1 Processor -- start 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.461+0000 7f4d2db81640 1 -- start start 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.461+0000 7f4d2db81640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28102d40 0x7f4d28198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.461+0000 7f4d2db81640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d28103660 0x7f4d28198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d2db81640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2810c170 0x7f4d2819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d2db81640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28111f70 con 0x7f4d28102d40 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d2db81640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28111df0 con 0x7f4d2810c170 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d2db81640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281120f0 con 0x7f4d28103660 2024-09-16T00:49:41.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28102d40 0x7f4d28198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28102d40 0x7f4d28198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35858/0 (socket says 172.21.15.31:35858) 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d277fe640 1 -- 172.21.15.31:0/3084571187 learned_addr learned my addr 172.21.15.31:0/3084571187 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d26ffd640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d28103660 0x7f4d28198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d27fff640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2810c170 0x7f4d2819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d277fe640 1 -- 172.21.15.31:0/3084571187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d28103660 msgr2=0x7f4d28198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d277fe640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d28103660 0x7f4d28198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d277fe640 1 -- 172.21.15.31:0/3084571187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2810c170 msgr2=0x7f4d2819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d277fe640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2810c170 0x7f4d2819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d277fe640 1 -- 172.21.15.31:0/3084571187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2819d3c0 con 0x7f4d28102d40 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d27fff640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2810c170 0x7f4d2819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d277fe640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28102d40 0x7f4d28198410 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800d950 tx=0x7f4d1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:41.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d24ff9640 1 -- 172.21.15.31:0/3084571187 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18017070 con 0x7f4d28102d40 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2819d6b0 con 0x7f4d28102d40 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d24ff9640 1 -- 172.21.15.31:0/3084571187 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d18009c30 con 0x7f4d28102d40 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.462+0000 7f4d24ff9640 1 -- 172.21.15.31:0/3084571187 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800f6b0 con 0x7f4d28102d40 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.463+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d281a4f90 con 0x7f4d28102d40 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.463+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d28103120 con 0x7f4d28102d40 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.465+0000 7f4d24ff9640 1 -- 172.21.15.31:0/3084571187 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4d180109d0 con 0x7f4d28102d40 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.465+0000 7f4d24ff9640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4d100a3240 0x7f4d100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.465+0000 7f4d26ffd640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4d100a3240 0x7f4d100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.466+0000 7f4d24ff9640 1 -- 172.21.15.31:0/3084571187 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9260+0+0 (secure 0 0 0) 0x7f4d180cf960 con 0x7f4d28102d40 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.466+0000 7f4d26ffd640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4d100a3240 0x7f4d100a5700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009230 tx=0x7f4d1c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:41.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.470+0000 7f4d24ff9640 1 -- 172.21.15.31:0/3084571187 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d18009de0 con 0x7f4d28102d40 2024-09-16T00:49:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:41 smithi081 ceph-mon[35539]: pgmap v613: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-16T00:49:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:41 smithi081 ceph-mon[35539]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:49:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:41 smithi081 ceph-mon[35539]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:49:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:41 smithi081 ceph-mon[35539]: Cluster is now healthy 2024-09-16T00:49:41.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:41 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/4119241796' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:49:41.498 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:41 smithi081 ceph-mon[35539]: fsmap 12 up:standby 2024-09-16T00:49:41.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:41.648+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4d28199770 con 0x7f4d28102d40 2024-09-16T00:49:42.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.023+0000 7f4d24ff9640 1 -- 172.21.15.31:0/3084571187 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f4d1809c280 con 0x7f4d28102d40 2024-09-16T00:49:42.024 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-09-16T00:49:42.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.027+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4d100a3240 msgr2=0x7f4d100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:42.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.027+0000 7f4d2db81640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4d100a3240 0x7f4d100a5700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009230 tx=0x7f4d1c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:49:42.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.027+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28102d40 msgr2=0x7f4d28198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:42.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.027+0000 7f4d2db81640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28102d40 0x7f4d28198410 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800d950 tx=0x7f4d1800de10 comp rx=0 tx=0).stop 2024-09-16T00:49:42.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.027+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 shutdown_connections 2024-09-16T00:49:42.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.028+0000 7f4d2db81640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2810c170 0x7f4d2819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:42.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.028+0000 7f4d2db81640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4d100a3240 0x7f4d100a5700 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:42.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.028+0000 7f4d2db81640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d28103660 0x7f4d28198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:42.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.028+0000 7f4d2db81640 1 --2- 172.21.15.31:0/3084571187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28102d40 0x7f4d28198410 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:42.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.028+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 >> 172.21.15.31:0/3084571187 conn(0x7f4d280fc7a0 msgr2=0x7f4d280fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:42.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.028+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 shutdown_connections 2024-09-16T00:49:42.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:42.028+0000 7f4d2db81640 1 -- 172.21.15.31:0/3084571187 wait complete. 2024-09-16T00:49:42.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:42 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3084571187' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:49:42.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:42 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3084571187' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:49:42.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:42 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3084571187' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:49:42.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T00:49:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:43.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:43 smithi031 ceph-mon[30956]: pgmap v614: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 307 B/s wr, 2 op/s 2024-09-16T00:49:43.125 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:43 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3084571187' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:49:43.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:43 smithi031 ceph-mon[30956]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:49:43.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:43 smithi102 ceph-mon[35364]: pgmap v614: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 307 B/s wr, 2 op/s 2024-09-16T00:49:43.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:43 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3084571187' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:49:43.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:43 smithi102 ceph-mon[35364]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:49:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:43 smithi081 ceph-mon[35539]: pgmap v614: 129 pgs: 129 active+clean; 189 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 307 B/s wr, 2 op/s 2024-09-16T00:49:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:43 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3084571187' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:49:43.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:43 smithi081 ceph-mon[35539]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:49:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.560+0000 7f4ba4793640 1 -- 172.21.15.31:0/624953727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c103660 msgr2=0x7f4b9c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.560+0000 7f4ba4793640 1 --2- 172.21.15.31:0/624953727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c103660 0x7f4b9c10bb50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009900 tx=0x7f4b9802a480 comp rx=0 tx=0).stop 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.560+0000 7f4ba4793640 1 -- 172.21.15.31:0/624953727 shutdown_connections 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.560+0000 7f4ba4793640 1 --2- 172.21.15.31:0/624953727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b9c10c170 0x7f4b9c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.560+0000 7f4ba4793640 1 --2- 172.21.15.31:0/624953727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c103660 0x7f4b9c10bb50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.560+0000 7f4ba4793640 1 --2- 172.21.15.31:0/624953727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b9c102d40 0x7f4b9c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.560+0000 7f4ba4793640 1 -- 172.21.15.31:0/624953727 >> 172.21.15.31:0/624953727 conn(0x7f4b9c0fc7a0 msgr2=0x7f4b9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.560+0000 7f4ba4793640 1 -- 172.21.15.31:0/624953727 shutdown_connections 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.560+0000 7f4ba4793640 1 -- 172.21.15.31:0/624953727 wait complete. 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba4793640 1 Processor -- start 2024-09-16T00:49:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba4793640 1 -- start start 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba4793640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c102d40 0x7f4b9c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba4793640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b9c103660 0x7f4b9c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba4793640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b9c10c170 0x7f4b9c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba4793640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c111fb0 con 0x7f4b9c102d40 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba4793640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c111e30 con 0x7f4b9c103660 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba4793640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c112130 con 0x7f4b9c10c170 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba252f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c102d40 0x7f4b9c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba252f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c102d40 0x7f4b9c198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35876/0 (socket says 172.21.15.31:35876) 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.561+0000 7f4ba252f640 1 -- 172.21.15.31:0/1054879442 learned_addr learned my addr 172.21.15.31:0/1054879442 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba2d30640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b9c10c170 0x7f4b9c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba1d2e640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b9c103660 0x7f4b9c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba252f640 1 -- 172.21.15.31:0/1054879442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b9c10c170 msgr2=0x7f4b9c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba252f640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b9c10c170 0x7f4b9c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba252f640 1 -- 172.21.15.31:0/1054879442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b9c103660 msgr2=0x7f4b9c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba252f640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b9c103660 0x7f4b9c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba252f640 1 -- 172.21.15.31:0/1054879442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c19d4f0 con 0x7f4b9c102d40 2024-09-16T00:49:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba252f640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c102d40 0x7f4b9c198540 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00b4a0 tx=0x7f4b8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4b937fe640 1 -- 172.21.15.31:0/1054879442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c014070 con 0x7f4b9c102d40 2024-09-16T00:49:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4b937fe640 1 -- 172.21.15.31:0/1054879442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b8c010af0 con 0x7f4b9c102d40 2024-09-16T00:49:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9c19d7e0 con 0x7f4b9c102d40 2024-09-16T00:49:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4b937fe640 1 -- 172.21.15.31:0/1054879442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c017d10 con 0x7f4b9c102d40 2024-09-16T00:49:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.562+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9c1a50c0 con 0x7f4b9c102d40 2024-09-16T00:49:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.563+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b9c103120 con 0x7f4b9c102d40 2024-09-16T00:49:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.564+0000 7f4b937fe640 1 -- 172.21.15.31:0/1054879442 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4b8c017070 con 0x7f4b9c102d40 2024-09-16T00:49:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.565+0000 7f4b937fe640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4b880a3240 0x7f4b880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.565+0000 7f4ba1d2e640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4b880a3240 0x7f4b880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.565+0000 7f4b937fe640 1 -- 172.21.15.31:0/1054879442 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8690+0+0 (secure 0 0 0) 0x7f4b8c0ceeb0 con 0x7f4b9c102d40 2024-09-16T00:49:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.566+0000 7f4ba1d2e640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4b880a3240 0x7f4b880a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0fee30 tx=0x7f4b98031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.570+0000 7f4b937fe640 1 -- 172.21.15.31:0/1054879442 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b8c09ba00 con 0x7f4b9c102d40 2024-09-16T00:49:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:43.749+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4b9c107240 con 0x7f4b9c102d40 2024-09-16T00:49:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.034+0000 7f4b937fe640 1 -- 172.21.15.31:0/1054879442 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f4b8c09b300 con 0x7f4b9c102d40 2024-09-16T00:49:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-09-16T00:49:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4b880a3240 msgr2=0x7f4b880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4b880a3240 0x7f4b880a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0fee30 tx=0x7f4b98031490 comp rx=0 tx=0).stop 2024-09-16T00:49:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c102d40 msgr2=0x7f4b9c198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c102d40 0x7f4b9c198540 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00b4a0 tx=0x7f4b8c00b960 comp rx=0 tx=0).stop 2024-09-16T00:49:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 shutdown_connections 2024-09-16T00:49:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7f4b880a3240 0x7f4b880a5700 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b9c10c170 0x7f4b9c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b9c103660 0x7f4b9c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 --2- 172.21.15.31:0/1054879442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b9c102d40 0x7f4b9c198540 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:44.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 >> 172.21.15.31:0/1054879442 conn(0x7f4b9c0fc7a0 msgr2=0x7f4b9c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:44.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 shutdown_connections 2024-09-16T00:49:44.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:44.036+0000 7f4ba4793640 1 -- 172.21.15.31:0/1054879442 wait complete. 2024-09-16T00:49:44.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:44 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1054879442' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:49:44.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:44 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1054879442' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:49:44.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:44 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1054879442' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:49:44.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-16T00:49:45.029 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:49:45.290 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:45 smithi031 ceph-mon[30956]: pgmap v616: 97 pgs: 97 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:49:45.290 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:45 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/1054879442' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:49:45.290 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:45 smithi031 ceph-mon[30956]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:49:45.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:45 smithi102 ceph-mon[35364]: pgmap v616: 97 pgs: 97 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:49:45.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:45 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/1054879442' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:49:45.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:45 smithi102 ceph-mon[35364]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:49:45.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:45 smithi081 ceph-mon[35539]: pgmap v616: 97 pgs: 97 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:49:45.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:45 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/1054879442' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:49:45.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:45 smithi081 ceph-mon[35539]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.530+0000 7fcd7b109640 1 -- 172.21.15.31:0/4169209045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 msgr2=0x7fcd741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.530+0000 7fcd7b109640 1 --2- 172.21.15.31:0/4169209045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 0x7fcd741067e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009900 tx=0x7fcd6002a410 comp rx=0 tx=0).stop 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.530+0000 7fcd7b109640 1 -- 172.21.15.31:0/4169209045 shutdown_connections 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.530+0000 7fcd7b109640 1 --2- 172.21.15.31:0/4169209045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd74106f10 0x7fcd7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.530+0000 7fcd7b109640 1 --2- 172.21.15.31:0/4169209045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 0x7fcd741067e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.530+0000 7fcd7b109640 1 --2- 172.21.15.31:0/4169209045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd74101de0 0x7fcd741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.530+0000 7fcd7b109640 1 -- 172.21.15.31:0/4169209045 >> 172.21.15.31:0/4169209045 conn(0x7fcd740fd8c0 msgr2=0x7fcd740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.530+0000 7fcd7b109640 1 -- 172.21.15.31:0/4169209045 shutdown_connections 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.530+0000 7fcd7b109640 1 -- 172.21.15.31:0/4169209045 wait complete. 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.531+0000 7fcd7b109640 1 Processor -- start 2024-09-16T00:49:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.531+0000 7fcd7b109640 1 -- start start 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd7b109640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd74101de0 0x7fcd741984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd7b109640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 0x7fcd741989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd7b109640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd74106f10 0x7fcd7419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd7b109640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74111fb0 con 0x7fcd74102790 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd7b109640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74111e30 con 0x7fcd74101de0 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd7b109640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74112130 con 0x7fcd74106f10 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd73fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 0x7fcd741989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd73fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 0x7fcd741989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35886/0 (socket says 172.21.15.31:35886) 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd73fff640 1 -- 172.21.15.31:0/3031370635 learned_addr learned my addr 172.21.15.31:0/3031370635 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd796a6640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd74106f10 0x7fcd7419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd78ea5640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd74101de0 0x7fcd741984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd73fff640 1 -- 172.21.15.31:0/3031370635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd74106f10 msgr2=0x7fcd7419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:45.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd73fff640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd74106f10 0x7fcd7419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd73fff640 1 -- 172.21.15.31:0/3031370635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd74101de0 msgr2=0x7fcd741984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd73fff640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd74101de0 0x7fcd741984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd73fff640 1 -- 172.21.15.31:0/3031370635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7419d460 con 0x7fcd74102790 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd78ea5640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd74101de0 0x7fcd741984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd73fff640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 0x7fcd741989f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcd60005ab0 tx=0x7fcd600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd71ffb640 1 -- 172.21.15.31:0/3031370635 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6003f030 con 0x7fcd74102790 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd71ffb640 1 -- 172.21.15.31:0/3031370635 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd60003ce0 con 0x7fcd74102790 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7419d6f0 con 0x7fcd74102790 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.532+0000 7fcd71ffb640 1 -- 172.21.15.31:0/3031370635 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60035070 con 0x7fcd74102790 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.533+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd741a5030 con 0x7fcd74102790 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.534+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd741021c0 con 0x7fcd74102790 2024-09-16T00:49:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.535+0000 7fcd71ffb640 1 -- 172.21.15.31:0/3031370635 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcd60042030 con 0x7fcd74102790 2024-09-16T00:49:45.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.535+0000 7fcd71ffb640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcd640a32c0 0x7fcd640a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:45.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.536+0000 7fcd78ea5640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcd640a32c0 0x7fcd640a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:45.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.536+0000 7fcd71ffb640 1 -- 172.21.15.31:0/3031370635 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8152+0+0 (secure 0 0 0) 0x7fcd600f01a0 con 0x7fcd74102790 2024-09-16T00:49:45.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.536+0000 7fcd78ea5640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcd640a32c0 0x7fcd640a5780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcd680058c0 tx=0x7fcd68005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:45.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.540+0000 7fcd71ffb640 1 -- 172.21.15.31:0/3031370635 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd600bcf60 con 0x7fcd74102790 2024-09-16T00:49:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:45.717+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcd74109bc0 con 0x7fcd74102790 2024-09-16T00:49:46.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.045+0000 7fcd71ffb640 1 -- 172.21.15.31:0/3031370635 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v95) ==== 178+0+0 (secure 0 0 0) 0x7fcd600bc860 con 0x7fcd74102790 2024-09-16T00:49:46.046 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data_ec' removed 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcd640a32c0 msgr2=0x7fcd640a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcd640a32c0 0x7fcd640a5780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcd680058c0 tx=0x7fcd68005d30 comp rx=0 tx=0).stop 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 msgr2=0x7fcd741989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 0x7fcd741989f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcd60005ab0 tx=0x7fcd600042f0 comp rx=0 tx=0).stop 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 shutdown_connections 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:6800/502885538,v1:172.21.15.31:6801/502885538] conn(0x7fcd640a32c0 0x7fcd640a5780 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd74106f10 0x7fcd7419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd74102790 0x7fcd741989f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 --2- 172.21.15.31:0/3031370635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd74101de0 0x7fcd741984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 >> 172.21.15.31:0/3031370635 conn(0x7fcd740fd8c0 msgr2=0x7fcd740feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 shutdown_connections 2024-09-16T00:49:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:49:46.049+0000 7fcd7b109640 1 -- 172.21.15.31:0/3031370635 wait complete. 2024-09-16T00:49:46.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:46 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3031370635' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:49:46.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:46 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3031370635' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:49:46.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:46 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3031370635' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:49:46.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:49:46.733 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:49:46.777 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:49:46.811 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T00:49:46.812 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:49:46.830 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:49:46.850 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:49:46.873 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T00:49:46.874 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T00:49:46.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.a 2024-09-16T00:49:47.299 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:47 smithi102 ceph-mon[35364]: pgmap v618: 33 pgs: 33 active+clean; 577 KiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:49:47.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:47 smithi102 ceph-mon[35364]: from='client.? 172.21.15.31:0/3031370635' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:49:47.300 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:47 smithi102 ceph-mon[35364]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T00:49:47.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:46 smithi031 systemd[1]: Stopping Ceph mon.a for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:49:47.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:47 smithi031 ceph-mon[30956]: pgmap v618: 33 pgs: 33 active+clean; 577 KiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:49:47.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:47 smithi031 ceph-mon[30956]: from='client.? 172.21.15.31:0/3031370635' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:49:47.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:47 smithi031 ceph-mon[30956]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T00:49:47.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:47 smithi081 ceph-mon[35539]: pgmap v618: 33 pgs: 33 active+clean; 577 KiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:49:47.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:47 smithi081 ceph-mon[35539]: from='client.? 172.21.15.31:0/3031370635' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:49:47.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:47 smithi081 ceph-mon[35539]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T00:49:47.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:47 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a[30952]: 2024-09-16T00:49:47.360+0000 7fcf99e21640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:49:47.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:47 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a[30952]: 2024-09-16T00:49:47.360+0000 7fcf99e21640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T00:49:47.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:47 smithi031 podman[99845]: 2024-09-16 00:49:47.513918477 +0000 UTC m=+0.420987065 container died 4b873356c0c186fc79a2931a7147f1b8aa118f993a84e7574bf8e5b6740d9d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T00:49:48.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:47 smithi031 podman[99845]: 2024-09-16 00:49:47.902471833 +0000 UTC m=+0.809540428 container cleanup 4b873356c0c186fc79a2931a7147f1b8aa118f993a84e7574bf8e5b6740d9d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T00:49:48.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:49:47 smithi031 bash[99845]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-a 2024-09-16T00:49:48.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T00:49:48.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:49:48.402 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T00:49:48.402 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-16T00:49:48.403 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.b 2024-09-16T00:49:48.740 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:48 smithi081 systemd[1]: Stopping Ceph mon.b for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:49:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:48 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-b[35535]: 2024-09-16T00:49:48.736+0000 7fe12c92b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:49:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:48 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-b[35535]: 2024-09-16T00:49:48.736+0000 7fe12c92b640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:49:48.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:48 smithi081 podman[68970]: 2024-09-16 00:49:48.787513381 +0000 UTC m=+0.174969925 container died ad386f878d0ac94fe642ebce62dade189c2ec2e22daabfc2cdeda80a6d905e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T00:49:49.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:49 smithi081 podman[68970]: 2024-09-16 00:49:49.204292336 +0000 UTC m=+0.591748886 container cleanup ad386f878d0ac94fe642ebce62dade189c2ec2e22daabfc2cdeda80a6d905e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-b, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T00:49:49.497 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:49 smithi081 bash[68970]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-b 2024-09-16T00:49:49.743 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T00:49:49.765 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:49 smithi081 podman[68982]: 2024-09-16 00:49:49.497679074 +0000 UTC m=+0.713153334 container remove ad386f878d0ac94fe642ebce62dade189c2ec2e22daabfc2cdeda80a6d905e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-b, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:49:49.765 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:49 smithi081 systemd[1]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-16T00:49:49.765 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:49 smithi081 systemd[1]: Stopped Ceph mon.b for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:49:49.765 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:49:49 smithi081 systemd[1]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.b.service: Consumed 10.509s CPU time. 2024-09-16T00:49:49.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:49:49.789 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-16T00:49:49.789 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-16T00:49:49.789 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.c 2024-09-16T00:49:50.109 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:49 smithi102 systemd[1]: Stopping Ceph mon.c for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:49:50.365 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:50 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-c[35360]: 2024-09-16T00:49:50.107+0000 7fa275681640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:49:50.365 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:50 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-c[35360]: 2024-09-16T00:49:50.107+0000 7fa275681640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:49:50.693 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:50 smithi102 podman[69109]: 2024-09-16 00:49:50.364653138 +0000 UTC m=+0.377967132 container died ebc8467ae9d62595d52b3a4d7d471e61aaaa98dc49e9bf9345be2f27248cb2c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:49:51.011 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:50 smithi102 podman[69109]: 2024-09-16 00:49:50.801515258 +0000 UTC m=+0.814829247 container cleanup ebc8467ae9d62595d52b3a4d7d471e61aaaa98dc49e9bf9345be2f27248cb2c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-16T00:49:51.011 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:49:50 smithi102 bash[69109]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mon-c 2024-09-16T00:49:51.231 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T00:49:51.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:49:51.276 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-16T00:49:51.276 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T00:49:51.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.x 2024-09-16T00:49:51.626 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:49:51 smithi031 systemd[1]: Stopping Ceph mgr.x for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:49:52.009 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:49:51 smithi031 podman[99964]: 2024-09-16 00:49:51.7227609 +0000 UTC m=+0.241883012 container died 509e457288037a18e446ff7e038bab5ec60bd6b96146c4698775424a52223407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T00:49:52.375 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:49:52 smithi031 podman[99964]: 2024-09-16 00:49:52.116510045 +0000 UTC m=+0.635632140 container cleanup 509e457288037a18e446ff7e038bab5ec60bd6b96146c4698775424a52223407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:49:52.376 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:49:52 smithi031 bash[99964]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-x 2024-09-16T00:49:52.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T00:49:52.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:49:52.554 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T00:49:52.554 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T00:49:52.554 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.y 2024-09-16T00:49:52.897 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:49:52 smithi081 systemd[1]: Stopping Ceph mgr.y for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:49:53.186 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:49:52 smithi081 podman[69097]: 2024-09-16 00:49:52.897249487 +0000 UTC m=+0.140225086 container died 51d2485aecbbd2a257573d4d9da6df6e17822e111cab671febde51426518e9f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:49:53.495 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:49:53 smithi081 podman[69097]: 2024-09-16 00:49:53.285163297 +0000 UTC m=+0.528138888 container cleanup 51d2485aecbbd2a257573d4d9da6df6e17822e111cab671febde51426518e9f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-16T00:49:53.495 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:49:53 smithi081 bash[69097]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-y 2024-09-16T00:49:53.704 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T00:49:53.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:49:53.738 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T00:49:53.738 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-16T00:49:53.738 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.z 2024-09-16T00:49:54.049 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:49:53 smithi102 systemd[1]: Stopping Ceph mgr.z for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:49:54.374 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:49:54 smithi102 podman[69232]: 2024-09-16 00:49:54.086092399 +0000 UTC m=+0.149112179 container died 2f053d2f087db7f9cb336cc9e4a736a6dd429a4f5b79446f62d294c7df6d97fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T00:49:54.759 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:49:54 smithi102 podman[69232]: 2024-09-16 00:49:54.474098711 +0000 UTC m=+0.537118484 container cleanup 2f053d2f087db7f9cb336cc9e4a736a6dd429a4f5b79446f62d294c7df6d97fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:49:54.759 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:49:54 smithi102 bash[69232]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mgr-z 2024-09-16T00:49:54.905 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-16T00:49:54.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:49:54.941 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-16T00:49:54.942 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T00:49:54.942 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.0 2024-09-16T00:49:55.288 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:49:54 smithi031 systemd[1]: Stopping Ceph osd.0 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:49:55.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:49:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0[42202]: 2024-09-16T00:49:55.288+0000 7f5532159640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:49:55.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:49:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0[42202]: 2024-09-16T00:49:55.288+0000 7f5532159640 -1 osd.0 95 *** Got signal Terminated *** 2024-09-16T00:49:55.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:49:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0[42202]: 2024-09-16T00:49:55.288+0000 7f5532159640 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:50:00.910 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:00 smithi031 podman[100084]: 2024-09-16 00:50:00.818497451 +0000 UTC m=+5.675978542 container died 39e78f2f169e180016340ecbbd9ce5af636c87a61523a5ad61e8e213630debf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:50:01.482 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:01 smithi031 podman[100084]: 2024-09-16 00:50:01.197071513 +0000 UTC m=+6.054552604 container cleanup 39e78f2f169e180016340ecbbd9ce5af636c87a61523a5ad61e8e213630debf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-16T00:50:01.482 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:01 smithi031 bash[100084]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0 2024-09-16T00:50:01.852 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:01 smithi031 podman[100104]: 2024-09-16 00:50:01.481965133 +0000 UTC m=+0.657757111 container remove 39e78f2f169e180016340ecbbd9ce5af636c87a61523a5ad61e8e213630debf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T00:50:02.119 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:01 smithi031 podman[100200]: 2024-09-16 00:50:01.757369112 +0000 UTC m=+0.018692661 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:50:02.119 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:01 smithi031 podman[100200]: 2024-09-16 00:50:01.866279121 +0000 UTC m=+0.127602658 container create 041a43f28c78df9c5918f0eeeb7dfa619fe2ee9d992834d45291534c8e746b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T00:50:02.376 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:02 smithi031 podman[100200]: 2024-09-16 00:50:02.11927151 +0000 UTC m=+0.380595056 container init 041a43f28c78df9c5918f0eeeb7dfa619fe2ee9d992834d45291534c8e746b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=) 2024-09-16T00:50:02.376 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:02 smithi031 podman[100200]: 2024-09-16 00:50:02.122927403 +0000 UTC m=+0.384250940 container start 041a43f28c78df9c5918f0eeeb7dfa619fe2ee9d992834d45291534c8e746b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-16T00:50:02.376 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:02 smithi031 podman[100200]: 2024-09-16 00:50:02.21092762 +0000 UTC m=+0.472251156 container attach 041a43f28c78df9c5918f0eeeb7dfa619fe2ee9d992834d45291534c8e746b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T00:50:02.376 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:50:02 smithi031 podman[100200]: 2024-09-16 00:50:02.315124716 +0000 UTC m=+0.576448258 container died 041a43f28c78df9c5918f0eeeb7dfa619fe2ee9d992834d45291534c8e746b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-16T00:50:02.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T00:50:03.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:50:03.014 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T00:50:03.014 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T00:50:03.014 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.1 2024-09-16T00:50:03.432 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:03 smithi081 systemd[1]: Stopping Ceph osd.1 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:50:03.747 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:03 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1[40003]: 2024-09-16T00:50:03.431+0000 7f3ffd9b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:50:03.747 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:03 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1[40003]: 2024-09-16T00:50:03.431+0000 7f3ffd9b2640 -1 osd.1 95 *** Got signal Terminated *** 2024-09-16T00:50:03.747 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:03 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1[40003]: 2024-09-16T00:50:03.431+0000 7f3ffd9b2640 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:50:09.247 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:08 smithi081 podman[69222]: 2024-09-16 00:50:08.993311939 +0000 UTC m=+5.777414611 container died b4d2e77149af41cd4edcfa53a8f974b165635718dce7e2c6c93e7329f0ce0ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:50:09.630 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:09 smithi081 podman[69222]: 2024-09-16 00:50:09.449208737 +0000 UTC m=+6.233311403 container cleanup b4d2e77149af41cd4edcfa53a8f974b165635718dce7e2c6c93e7329f0ce0ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:50:09.630 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:09 smithi081 bash[69222]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1 2024-09-16T00:50:10.009 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:09 smithi081 podman[69237]: 2024-09-16 00:50:09.684584002 +0000 UTC m=+0.693802344 container remove b4d2e77149af41cd4edcfa53a8f974b165635718dce7e2c6c93e7329f0ce0ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T00:50:10.446 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:10 smithi081 podman[69336]: 2024-09-16 00:50:10.051059486 +0000 UTC m=+0.016307165 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:50:10.446 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:10 smithi081 podman[69336]: 2024-09-16 00:50:10.185544457 +0000 UTC m=+0.150792128 container create ce96dc8ab5cd8596b77728fc7bda73048be7f1b154112f256b8fbd6bee3b3cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:50:10.447 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:10 smithi081 podman[69336]: 2024-09-16 00:50:10.446676205 +0000 UTC m=+0.411923886 container init ce96dc8ab5cd8596b77728fc7bda73048be7f1b154112f256b8fbd6bee3b3cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:50:10.747 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:10 smithi081 podman[69336]: 2024-09-16 00:50:10.450503177 +0000 UTC m=+0.415750848 container start ce96dc8ab5cd8596b77728fc7bda73048be7f1b154112f256b8fbd6bee3b3cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:50:10.748 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:10 smithi081 podman[69336]: 2024-09-16 00:50:10.546761465 +0000 UTC m=+0.512009148 container attach ce96dc8ab5cd8596b77728fc7bda73048be7f1b154112f256b8fbd6bee3b3cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T00:50:10.748 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:50:10 smithi081 podman[69336]: 2024-09-16 00:50:10.677106061 +0000 UTC m=+0.642353733 container died ce96dc8ab5cd8596b77728fc7bda73048be7f1b154112f256b8fbd6bee3b3cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:50:11.398 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T00:50:11.437 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:50:11.438 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T00:50:11.438 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T00:50:11.438 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.2 2024-09-16T00:50:11.763 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:11 smithi102 systemd[1]: Stopping Ceph osd.2 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:50:12.049 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:11 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2[40340]: 2024-09-16T00:50:11.762+0000 7fe6d2d48640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:50:12.049 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:11 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2[40340]: 2024-09-16T00:50:11.762+0000 7fe6d2d48640 -1 osd.2 95 *** Got signal Terminated *** 2024-09-16T00:50:12.050 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:11 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2[40340]: 2024-09-16T00:50:11.762+0000 7fe6d2d48640 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:50:17.430 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:17 smithi102 podman[69357]: 2024-09-16 00:50:17.124262545 +0000 UTC m=+5.484667060 container died 3052100a338e04c58b54924c9966bc28e2ec22388488e27c50ac88fe2623ca0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:50:17.781 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:17 smithi102 podman[69357]: 2024-09-16 00:50:17.521102052 +0000 UTC m=+5.881506567 container cleanup 3052100a338e04c58b54924c9966bc28e2ec22388488e27c50ac88fe2623ca0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:50:17.781 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:17 smithi102 bash[69357]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2 2024-09-16T00:50:18.049 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:17 smithi102 podman[69374]: 2024-09-16 00:50:17.781206797 +0000 UTC m=+0.653313813 container remove 3052100a338e04c58b54924c9966bc28e2ec22388488e27c50ac88fe2623ca0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:50:18.393 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:18 smithi102 podman[69463]: 2024-09-16 00:50:18.033967269 +0000 UTC m=+0.015688360 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:50:18.394 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:18 smithi102 podman[69463]: 2024-09-16 00:50:18.14876735 +0000 UTC m=+0.130488429 container create 49bad9fd2476f679db6a71e3dea6f593dfc3a35b305b6003cd0217c90b6e3147 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-16T00:50:18.394 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:18 smithi102 podman[69463]: 2024-09-16 00:50:18.393509243 +0000 UTC m=+0.375230320 container init 49bad9fd2476f679db6a71e3dea6f593dfc3a35b305b6003cd0217c90b6e3147 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-16T00:50:18.800 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:18 smithi102 podman[69463]: 2024-09-16 00:50:18.39702401 +0000 UTC m=+0.378745094 container start 49bad9fd2476f679db6a71e3dea6f593dfc3a35b305b6003cd0217c90b6e3147 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T00:50:18.800 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:18 smithi102 podman[69463]: 2024-09-16 00:50:18.510253092 +0000 UTC m=+0.491974173 container attach 49bad9fd2476f679db6a71e3dea6f593dfc3a35b305b6003cd0217c90b6e3147 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:50:18.800 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:50:18 smithi102 podman[69463]: 2024-09-16 00:50:18.585113265 +0000 UTC m=+0.566834354 container died 49bad9fd2476f679db6a71e3dea6f593dfc3a35b305b6003cd0217c90b6e3147 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T00:50:19.270 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T00:50:19.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:50:19.309 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T00:50:19.309 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T00:50:19.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.3 2024-09-16T00:50:19.626 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:19 smithi031 systemd[1]: Stopping Ceph osd.3 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:50:20.125 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:19 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3[49114]: 2024-09-16T00:50:19.649+0000 7ff46a26b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:50:20.126 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:19 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3[49114]: 2024-09-16T00:50:19.649+0000 7ff46a26b640 -1 osd.3 95 *** Got signal Terminated *** 2024-09-16T00:50:20.126 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:19 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3[49114]: 2024-09-16T00:50:19.649+0000 7ff46a26b640 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:50:23.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:23 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:23.196+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:24.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:24.159+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:24.876 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:24 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3[49114]: 2024-09-16T00:50:24.477+0000 7ff462879640 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.615762+0000 front 2024-09-16T00:49:59.615786+0000 (oldest deadline 2024-09-16T00:50:23.715866+0000) 2024-09-16T00:50:25.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:25.184+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:25.585 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:25 smithi031 podman[100305]: 2024-09-16 00:50:25.182895008 +0000 UTC m=+5.672080410 container died 51f5087630add2a2b0cefd44830a6c11b4381b03b334b1450dc239d7b327163a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T00:50:25.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:25 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:25.372+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:25.875 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:25 smithi031 podman[100305]: 2024-09-16 00:50:25.792630628 +0000 UTC m=+6.281816029 container cleanup 51f5087630add2a2b0cefd44830a6c11b4381b03b334b1450dc239d7b327163a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:50:25.876 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:25 smithi031 bash[100305]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3 2024-09-16T00:50:26.299 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:26 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:26.013+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:26.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:26 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:26.167+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:26.376 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:26 smithi031 podman[100321]: 2024-09-16 00:50:26.077573674 +0000 UTC m=+0.887960023 container remove 51f5087630add2a2b0cefd44830a6c11b4381b03b334b1450dc239d7b327163a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T00:50:26.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:26 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:26.414+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:26.797 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:26 smithi031 podman[100406]: 2024-09-16 00:50:26.3001997 +0000 UTC m=+0.016646883 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:50:26.797 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:26 smithi031 podman[100406]: 2024-09-16 00:50:26.428534298 +0000 UTC m=+0.144981523 container create d63b3f529ef0d11a211fbd83c2fef4bfbfeac2a89cedbc487a2d43418bb48073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T00:50:26.797 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:26 smithi031 podman[100406]: 2024-09-16 00:50:26.797473066 +0000 UTC m=+0.513920249 container init d63b3f529ef0d11a211fbd83c2fef4bfbfeac2a89cedbc487a2d43418bb48073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T00:50:27.126 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:26 smithi031 podman[100406]: 2024-09-16 00:50:26.801446879 +0000 UTC m=+0.517894062 container start d63b3f529ef0d11a211fbd83c2fef4bfbfeac2a89cedbc487a2d43418bb48073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T00:50:27.126 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:26 smithi031 podman[100406]: 2024-09-16 00:50:26.931975995 +0000 UTC m=+0.648423180 container attach d63b3f529ef0d11a211fbd83c2fef4bfbfeac2a89cedbc487a2d43418bb48073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T00:50:27.126 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:50:27 smithi031 podman[100429]: 2024-09-16 00:50:27.029280167 +0000 UTC m=+0.020220699 container died d63b3f529ef0d11a211fbd83c2fef4bfbfeac2a89cedbc487a2d43418bb48073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T00:50:27.549 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:27.060+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:27.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:27.134+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:27.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T00:50:27.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:50:27.663 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T00:50:27.663 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T00:50:27.663 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.4 2024-09-16T00:50:27.690 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:27 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:27.428+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:27.997 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:27 smithi081 systemd[1]: Stopping Ceph osd.4 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:50:28.404 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:28 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4[46602]: 2024-09-16T00:50:28.003+0000 7f7e0b7f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:50:28.404 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:28 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4[46602]: 2024-09-16T00:50:28.003+0000 7f7e0b7f9640 -1 osd.4 95 *** Got signal Terminated *** 2024-09-16T00:50:28.404 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:28 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4[46602]: 2024-09-16T00:50:28.003+0000 7f7e0b7f9640 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:50:28.549 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:28.103+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:28.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:28.159+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:28.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:28 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:28.403+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:29.549 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:29.152+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:29.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:29.186+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:29.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:29 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:29.407+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:30.299 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:30 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:30.153+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:30.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:30 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:30.145+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:30.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:30 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:30.362+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:30.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:30 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:30.362+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:31.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:30 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:30.696+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:31.552 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:31 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:31.168+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:31.553 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:31 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:31.138+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:31.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:31 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:31.485+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:31.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:31 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:31.394+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:31.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:31 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:31.394+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:32.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:31 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:31.721+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:32.436 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:32 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4[46602]: 2024-09-16T00:50:32.010+0000 7f7e07e0f640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.376023+0000 front 2024-09-16T00:50:07.375948+0000 (oldest deadline 2024-09-16T00:50:31.476057+0000) 2024-09-16T00:50:32.549 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:32 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:32.130+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:32.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:32 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:32.137+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:32.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:32 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:32.515+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:32.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:32 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:32.436+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:32.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:32 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:32.436+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:33.125 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:32 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:32.770+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:33.398 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:33 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4[46602]: 2024-09-16T00:50:33.000+0000 7f7e07e0f640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.376023+0000 front 2024-09-16T00:50:07.375948+0000 (oldest deadline 2024-09-16T00:50:31.476057+0000) 2024-09-16T00:50:33.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:33 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:33.180+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:33.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:33 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:33.153+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:33.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:33 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:33.153+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.914879+0000 front 2024-09-16T00:50:07.914692+0000 (oldest deadline 2024-09-16T00:50:32.614736+0000) 2024-09-16T00:50:33.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:33 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:33.497+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:33.748 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:33 smithi081 podman[69441]: 2024-09-16 00:50:33.465637217 +0000 UTC m=+5.599224799 container died 5a4a49fcf8070647544d65e924f45e65ce79486d3525dd1fadcf2f3611dbdd8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:50:33.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:33 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:33.398+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:33.749 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:33 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:33.398+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:33.773 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:33 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:33.435+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:34.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:33 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:33.773+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:34.214 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:33 smithi081 podman[69441]: 2024-09-16 00:50:33.854054878 +0000 UTC m=+5.987642453 container cleanup 5a4a49fcf8070647544d65e924f45e65ce79486d3525dd1fadcf2f3611dbdd8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T00:50:34.214 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:33 smithi081 bash[69441]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4 2024-09-16T00:50:34.497 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:34 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:34.447+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:34.497 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:34 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:34.406+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:34.497 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:34 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:34.406+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:34.498 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:34 smithi081 podman[69457]: 2024-09-16 00:50:34.21405619 +0000 UTC m=+0.741996203 container remove 5a4a49fcf8070647544d65e924f45e65ce79486d3525dd1fadcf2f3611dbdd8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:50:34.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:34 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:34.215+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:34.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:34 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:34.155+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:34.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:34 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:34.155+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.914879+0000 front 2024-09-16T00:50:07.914692+0000 (oldest deadline 2024-09-16T00:50:32.614736+0000) 2024-09-16T00:50:34.776 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:34 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:34.469+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:34.909 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:34 smithi081 podman[69546]: 2024-09-16 00:50:34.522848193 +0000 UTC m=+0.015575914 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:50:34.909 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:34 smithi081 podman[69546]: 2024-09-16 00:50:34.640139585 +0000 UTC m=+0.132867291 container create 429d1f029981b634fac5ab51b3b8236701b4fb346cbba33a092a7c915e49f8b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:50:35.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:34 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:34.775+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:35.248 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:34 smithi081 podman[69546]: 2024-09-16 00:50:34.90947878 +0000 UTC m=+0.402206487 container init 429d1f029981b634fac5ab51b3b8236701b4fb346cbba33a092a7c915e49f8b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T00:50:35.248 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:34 smithi081 podman[69546]: 2024-09-16 00:50:34.912780107 +0000 UTC m=+0.405507829 container start 429d1f029981b634fac5ab51b3b8236701b4fb346cbba33a092a7c915e49f8b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:50:35.248 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:35 smithi081 podman[69546]: 2024-09-16 00:50:35.017898335 +0000 UTC m=+0.510626047 container attach 429d1f029981b634fac5ab51b3b8236701b4fb346cbba33a092a7c915e49f8b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T00:50:35.248 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:50:35 smithi081 podman[69546]: 2024-09-16 00:50:35.100123008 +0000 UTC m=+0.592850734 container died 429d1f029981b634fac5ab51b3b8236701b4fb346cbba33a092a7c915e49f8b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T00:50:35.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:35 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:35.399+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:35.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:35 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:35.257+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:35.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:35 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:35.198+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:35.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:35 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:35.198+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.914879+0000 front 2024-09-16T00:50:07.914692+0000 (oldest deadline 2024-09-16T00:50:32.614736+0000) 2024-09-16T00:50:35.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:35 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:35.431+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:35.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:35 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:35.439+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:35.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:35 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:35.439+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:35.813 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T00:50:35.875 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:35.514+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:35.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:35 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:35.760+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:35.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:50:35.893 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T00:50:35.893 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T00:50:35.893 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.5 2024-09-16T00:50:36.202 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:35 smithi102 systemd[1]: Stopping Ceph osd.5 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:50:36.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:36 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:36.392+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:36.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:36 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:36.285+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:36.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:36 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:36.200+0000 7f4ea2cca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:50:36.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:36 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:36.200+0000 7f4ea2cca640 -1 osd.5 95 *** Got signal Terminated *** 2024-09-16T00:50:36.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:36 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:36.200+0000 7f4ea2cca640 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:50:36.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:36 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:36.222+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:36.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:36 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:36.222+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.914879+0000 front 2024-09-16T00:50:07.914692+0000 (oldest deadline 2024-09-16T00:50:32.614736+0000) 2024-09-16T00:50:36.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:36 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:36.408+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:36.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:36 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:36.412+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:36.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:36 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:36.412+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:36.875 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:36 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:36.538+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:36.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:36 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:36.726+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:37.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:37 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:37.389+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:37.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:37 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:37.244+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:37.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:37 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:37.230+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:37.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:37 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:37.230+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.914879+0000 front 2024-09-16T00:50:07.914692+0000 (oldest deadline 2024-09-16T00:50:32.614736+0000) 2024-09-16T00:50:37.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:37 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:37.432+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:37.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:37 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:37.432+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:37.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:37 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:37.435+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:37.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:37 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:37.435+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:37.875 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:37.500+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:37.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:37.699+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:38.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:38 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:38.358+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:38.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:38 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:38.219+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:38.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:38 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:38.203+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:38.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:38 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:38.203+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.914879+0000 front 2024-09-16T00:50:07.914692+0000 (oldest deadline 2024-09-16T00:50:32.614736+0000) 2024-09-16T00:50:38.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:38 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:38.421+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:38.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:38 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:38.421+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:38.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:38 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:38.469+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:38.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:38 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:38.469+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:38.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:38 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:38.469+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:38.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:38.502+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:38.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:38.502+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:38.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:38.748+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:38.877 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:38 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:38.748+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:39.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:39 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:39.183+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:39.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:39 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:39.320+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:39.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:39 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:39.184+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:39.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:39 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:39.184+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.914879+0000 front 2024-09-16T00:50:07.914692+0000 (oldest deadline 2024-09-16T00:50:32.614736+0000) 2024-09-16T00:50:39.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:39 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:39.184+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.515548+0000 front 2024-09-16T00:50:15.515506+0000 (oldest deadline 2024-09-16T00:50:39.015665+0000) 2024-09-16T00:50:39.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:39 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:39.430+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:39.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:39 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:39.430+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:39.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:39 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:39.479+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:39.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:39 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:39.479+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:39.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:39 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:39.479+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:39.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:39 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:39.496+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:39.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:39 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:39.496+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:39.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:39 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:39.706+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:39.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:39 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:39.706+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:40.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:40 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:40.282+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:40.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:40 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:40.175+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:40.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:40 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:40.233+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.713983+0000 front 2024-09-16T00:49:59.714078+0000 (oldest deadline 2024-09-16T00:50:25.613774+0000) 2024-09-16T00:50:40.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:40 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:40.233+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:07.914879+0000 front 2024-09-16T00:50:07.914692+0000 (oldest deadline 2024-09-16T00:50:32.614736+0000) 2024-09-16T00:50:40.550 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:40 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5[46816]: 2024-09-16T00:50:40.233+0000 7f4e9b2d8640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.515548+0000 front 2024-09-16T00:50:15.515506+0000 (oldest deadline 2024-09-16T00:50:39.015665+0000) 2024-09-16T00:50:40.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:40 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:40.450+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:40.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:40 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:40.450+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:40.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:40 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:40.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:40.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:40 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:40.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:40.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:40 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:40.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:40.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:40 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:40.544+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:40.958 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:40 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:40.544+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:40.958 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:40 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:40.707+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:40.958 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:40 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:40.707+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:41.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:41 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:41.284+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:41.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:41 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:41.284+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:41.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:41 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:41.131+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:41.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:41 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:41.131+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:41.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:41 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:41.419+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:41.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:41 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:41.419+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:41.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:41 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:41.494+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:41.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:41 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:41.494+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:41.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:41 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:41.494+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:41.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:41 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:41.564+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:41.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:41 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:41.564+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:41.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:41 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:41.691+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:41.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:41 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:41.691+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:42.011 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:41 smithi102 podman[69578]: 2024-09-16 00:50:41.735012746 +0000 UTC m=+5.647312487 container died 3d271fdffb314ed4eacacad589fd32a51cd46b37ae603e9cd0752100a39a252d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:50:42.300 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:42 smithi102 podman[69578]: 2024-09-16 00:50:42.101267446 +0000 UTC m=+6.013567183 container cleanup 3d271fdffb314ed4eacacad589fd32a51cd46b37ae603e9cd0752100a39a252d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T00:50:42.300 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:42 smithi102 bash[69578]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5 2024-09-16T00:50:42.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:42 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:42.132+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:42.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:42 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:42.132+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:42.300 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:42 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:42.240+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:42.301 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:42 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:42.240+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:42.728 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:42 smithi102 podman[69593]: 2024-09-16 00:50:42.369392681 +0000 UTC m=+0.632756196 container remove 3d271fdffb314ed4eacacad589fd32a51cd46b37ae603e9cd0752100a39a252d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:50:42.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:42 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:42.427+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:42.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:42 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:42.427+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:42.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:42 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:42.446+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:42.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:42 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:42.446+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:42.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:42 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:42.446+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:42.875 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:42 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:42.532+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:42.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:42 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:42.532+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:42.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:42 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:42.681+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:42.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:42 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:42.681+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:43.006 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:42 smithi102 podman[69684]: 2024-09-16 00:50:42.630866695 +0000 UTC m=+0.016071944 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:50:43.006 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:42 smithi102 podman[69684]: 2024-09-16 00:50:42.736910515 +0000 UTC m=+0.122115766 container create 18908bbdb00d50e42185390cd92ace2299039d40ed2b948db5f84cbf418dbc64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:50:43.299 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:43 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:43.158+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:43.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:43 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:43.158+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:43.300 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:43 smithi102 podman[69684]: 2024-09-16 00:50:43.006396988 +0000 UTC m=+0.391602235 container init 18908bbdb00d50e42185390cd92ace2299039d40ed2b948db5f84cbf418dbc64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T00:50:43.300 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:43 smithi102 podman[69684]: 2024-09-16 00:50:43.009742428 +0000 UTC m=+0.394947687 container start 18908bbdb00d50e42185390cd92ace2299039d40ed2b948db5f84cbf418dbc64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:50:43.300 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:43 smithi102 podman[69684]: 2024-09-16 00:50:43.106579542 +0000 UTC m=+0.491784802 container attach 18908bbdb00d50e42185390cd92ace2299039d40ed2b948db5f84cbf418dbc64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T00:50:43.300 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:50:43 smithi102 podman[69684]: 2024-09-16 00:50:43.198013718 +0000 UTC m=+0.583218966 container died 18908bbdb00d50e42185390cd92ace2299039d40ed2b948db5f84cbf418dbc64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e) 2024-09-16T00:50:43.301 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:43 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:43.238+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:43.301 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:43 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:43.238+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:43.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:43 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:43.473+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:43.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:43 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:43.473+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:43.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:43 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:43.488+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:43.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:43 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:43.488+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:43.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:43 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:43.488+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:43.862 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T00:50:43.875 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:43 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:43.557+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:43.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:43 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:43.557+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:43.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:43 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:43.638+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:43.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:43 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:43.638+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:43.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:50:43.906 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T00:50:43.906 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T00:50:43.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.6 2024-09-16T00:50:44.221 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:43 smithi031 systemd[1]: Stopping Ceph osd.6 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:50:44.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:44 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:44.206+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:44.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:44 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:44.206+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:44.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:44 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:44.176+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:44.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:44 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:44.176+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:44.595 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:44 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:44.220+0000 7f5420d37640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:50:44.596 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:44 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:44.220+0000 7f5420d37640 -1 osd.6 95 *** Got signal Terminated *** 2024-09-16T00:50:44.596 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:44 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:44.220+0000 7f5420d37640 -1 osd.6 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:50:44.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:44 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:44.443+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:44.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:44 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:44.443+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:44.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:44 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:44.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:44.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:44 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:44.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:44.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:44 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:44.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:44.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:44 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:44.595+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:44.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:44 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:44.595+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:44.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:44 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:44.665+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:44.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:44 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:44.665+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:45.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:45 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:45.186+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:45.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:45 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:45.186+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:45.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:45 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:45.225+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:45.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:45 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:45.225+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:45.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:45 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:45.402+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:45.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:45 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:45.402+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:45.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:45 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:45.501+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:45.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:45 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:45.501+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:45.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:45 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:45.501+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:45.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:45 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:45.567+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:45.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:45 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:45.567+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:45.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:45 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:45.708+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:45.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:45 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:45.708+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:46.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:46 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:46.152+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:46.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:46 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:46.152+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:46.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:46 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:46.230+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:46.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:46 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:46.230+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:46.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:46 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:46.362+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:46.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:46 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:46.362+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:46.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:46 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:46.362+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:46.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:46 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:46.496+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:46.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:46 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:46.496+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:46.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:46 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:46.496+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:46.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:46 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:46.602+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:46.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:46 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:46.602+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:46.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:46 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:46.712+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:46.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:46 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:46.712+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:46.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:46 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:46.712+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:47.451 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:47 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:47.136+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:47.451 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:47 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:47.136+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:47.451 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:47 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:47.136+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:47.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:47 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:47.199+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:47.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:47 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:47.199+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:47.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:47 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:47.199+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:47.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:47 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:47.337+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:47.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:47 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:47.337+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:47.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:47 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:47.337+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:47.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:47 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:47.494+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:47.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:47 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:47.494+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:47.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:47 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:47.494+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:47.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:47 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:47.494+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:47.875 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:47 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:47.566+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:47.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:47 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:47.566+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:47.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:47 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:47.673+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:47.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:47 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:47.673+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:47.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:47 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:47.673+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:48.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:48 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:48.170+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:48.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:48 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:48.170+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:48.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:48 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:48.170+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:48.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:48 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:48.137+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:48.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:48 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:48.137+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:48.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:48 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:48.137+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:48.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:48 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:48.370+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:48.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:48 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:48.370+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:48.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:48 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:48.370+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:48.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:48 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:48.518+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:48.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:48 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:48.518+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:48.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:48 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:48.518+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:48.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:48 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:48.518+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:48.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:48 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:48.580+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:03.199404+0000 front 2024-09-16T00:50:03.199451+0000 (oldest deadline 2024-09-16T00:50:33.200260+0000) 2024-09-16T00:50:48.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:48 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:48.580+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.901222+0000 front 2024-09-16T00:50:14.901251+0000 (oldest deadline 2024-09-16T00:50:37.801278+0000) 2024-09-16T00:50:48.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:48 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6[55600]: 2024-09-16T00:50:48.580+0000 7f541d34d640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:23.503306+0000 front 2024-09-16T00:50:23.503321+0000 (oldest deadline 2024-09-16T00:50:47.603496+0000) 2024-09-16T00:50:48.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:48 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:48.686+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:48.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:48 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:48.686+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:48.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:48 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:48.686+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:49.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:49 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:49.184+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:49.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:49 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:49.184+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:49.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:49 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:49.184+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:49.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:49 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:49.208+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:49.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:49 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:49.208+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:49.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:49 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:49.208+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:49.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:49 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:49.350+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:49.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:49 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:49.350+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:49.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:49 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:49.350+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:49.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:49 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:49.476+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:49.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:49 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:49.476+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:49.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:49 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:49.476+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:49.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:49 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:49.476+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:49.876 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:49 smithi031 podman[100521]: 2024-09-16 00:50:49.576997484 +0000 UTC m=+5.467254052 container died 00388baf15ea4cfd2f466d76a12361054ee1dc3e3a5c697c8b18ebf89b40cf5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:50:49.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:49 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:49.725+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:49.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:49 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:49.725+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:49.876 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:49 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:49.725+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:50.376 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:50 smithi031 podman[100521]: 2024-09-16 00:50:50.228995017 +0000 UTC m=+6.119251630 container cleanup 00388baf15ea4cfd2f466d76a12361054ee1dc3e3a5c697c8b18ebf89b40cf5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:50:50.376 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:50 smithi031 bash[100521]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6 2024-09-16T00:50:50.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:50 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:50.214+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:50.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:50 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:50.214+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:50.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:50 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:50.214+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:50.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:50 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:50.247+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:50.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:50 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:50.247+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:50.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:50 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:50.247+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:50.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:50 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:50.393+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:50.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:50 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:50.393+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:50.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:50 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:50.393+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:50.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:50 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:50.456+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:50.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:50 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:50.456+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:50.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:50 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:50.456+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:50.749 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:50 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:50.456+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:50.755 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:50 smithi031 podman[100536]: 2024-09-16 00:50:50.480568709 +0000 UTC m=+0.903097629 container remove 00388baf15ea4cfd2f466d76a12361054ee1dc3e3a5c697c8b18ebf89b40cf5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:50:51.042 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:50 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:50.755+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:51.042 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:50 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:50.755+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:51.042 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:50 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:50.755+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:51.043 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:50 smithi031 podman[100620]: 2024-09-16 00:50:50.789639874 +0000 UTC m=+0.099408239 container create 14ad27f493eef36806a1facf39bbe614d7c099f509f477abda72ed24bcf5ea37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:50:51.043 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:50 smithi031 podman[100620]: 2024-09-16 00:50:50.706746216 +0000 UTC m=+0.016514529 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:50:51.043 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:51 smithi031 podman[100620]: 2024-09-16 00:50:51.042248149 +0000 UTC m=+0.352016453 container init 14ad27f493eef36806a1facf39bbe614d7c099f509f477abda72ed24bcf5ea37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:50:51.376 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:51 smithi031 podman[100620]: 2024-09-16 00:50:51.04607587 +0000 UTC m=+0.355844173 container start 14ad27f493eef36806a1facf39bbe614d7c099f509f477abda72ed24bcf5ea37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T00:50:51.376 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:51 smithi031 podman[100620]: 2024-09-16 00:50:51.117165576 +0000 UTC m=+0.426933892 container attach 14ad27f493eef36806a1facf39bbe614d7c099f509f477abda72ed24bcf5ea37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:50:51.376 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:50:51 smithi031 podman[100620]: 2024-09-16 00:50:51.240080685 +0000 UTC m=+0.549848994 container died 14ad27f493eef36806a1facf39bbe614d7c099f509f477abda72ed24bcf5ea37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:50:51.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:51 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:51.230+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:51.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:51 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:51.230+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:51.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:51 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:51.230+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:51.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:51 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:51.218+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:51.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:51 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:51.218+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:51.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:51 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:51.218+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:51.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:51 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:51.413+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:51.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:51 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:51.413+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:51.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:51 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:51.413+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:51.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:51 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:51.488+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:51.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:51 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:51.488+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:51.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:51 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:51.488+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:51.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:51 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:51.488+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:51.996 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:51 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:51.715+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:51.996 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:51 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:51.715+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:51.996 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:51 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:51.715+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:52.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T00:50:52.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:50:52.065 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T00:50:52.065 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T00:50:52.065 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.7 2024-09-16T00:50:52.367 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:52 smithi081 systemd[1]: Stopping Ceph osd.7 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:50:52.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:52 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:52.229+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:52.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:52 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:52.229+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:52.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:52 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:52.229+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:52.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:52 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:52.209+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:52.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:52 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:52.209+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:52.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:52 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:52.209+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:52.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:52.441+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:52.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:52.441+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:52.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:52.441+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:52.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:52.441+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:52.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:52.366+0000 7f3113513640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:50:52.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:52.366+0000 7f3113513640 -1 osd.7 95 *** Got signal Terminated *** 2024-09-16T00:50:52.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:52.366+0000 7f3113513640 -1 osd.7 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:50:52.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:52.442+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:52.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:52.442+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:52.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:52 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:52.442+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:53.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:52 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:52.687+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:53.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:52 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:52.687+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:53.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:52 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:52.687+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:53.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:53.238+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:53.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:53.238+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:53.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:53.238+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:53.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:53.189+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:53.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:53.189+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:53.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:53 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:53.189+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:53.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:53 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:53.401+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:53.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:53 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:53.401+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:53.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:53 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:53.401+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:53.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:53 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:53.432+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:53.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:53 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:53.432+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:53.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:53 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:53.432+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:53.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:53 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:53.432+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:54.125 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:53 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:53.684+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:54.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:53 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:53.684+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:54.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:53 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:53.684+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:54.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:54 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:54.232+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:54.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:54 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:54.232+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:54.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:54 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:54.232+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:54.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:54 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:54.188+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:54.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:54 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:54.188+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:54.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:54 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:54.188+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:54.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:54 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:54.188+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:50:54.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:54 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:54.355+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:54.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:54 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:54.355+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:54.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:54 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:54.355+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:54.749 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:54 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:54.445+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:54.749 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:54 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:54.445+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:54.749 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:54 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:54.445+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:54.750 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:54 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:54.445+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:54.750 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:54 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:54.445+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:50:55.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:54 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:54.644+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:55.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:54 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:54.644+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:55.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:54 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:54.644+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:55.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:55.243+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:55.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:55.243+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:55.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:55.243+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:55.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:55.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:55.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:55.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:55.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:55.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:55.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:55 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:55.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:50:55.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:55.382+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:55.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:55.382+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:55.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:55.382+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:55.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:55.491+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:55.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:55.491+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:55.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:55.491+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:55.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:55.491+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:55.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:55.491+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:50:56.125 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:55.682+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:56.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:55.682+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:56.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:55 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:55.682+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:56.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:56.215+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:56.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:56.215+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:56.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:56.215+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:56.551 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:56.191+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:56.551 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:56.191+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:56.551 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:56.191+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:56.551 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:56 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:56.191+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:50:56.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:56.348+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:56.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:56.348+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:56.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:56.348+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:56.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:56.528+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:56.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:56.528+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:56.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:56.528+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:56.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:56.528+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:56.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:56 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:56.528+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:50:57.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:56.702+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:57.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:56.702+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:57.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:56 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:56.702+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:57.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:57.188+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:57.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:57.188+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:57.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:57.188+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:57.551 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:57.188+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.409990+0000 front 2024-09-16T00:50:32.409956+0000 (oldest deadline 2024-09-16T00:50:56.509910+0000) 2024-09-16T00:50:57.551 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:57.173+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:57.551 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:57.173+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:57.551 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:57.173+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:57.551 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:57 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:57.173+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:50:57.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:57.337+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:05.939075+0000 front 2024-09-16T00:50:05.939052+0000 (oldest deadline 2024-09-16T00:50:31.239158+0000) 2024-09-16T00:50:57.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:57.337+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:11.239695+0000 front 2024-09-16T00:50:11.239773+0000 (oldest deadline 2024-09-16T00:50:37.139626+0000) 2024-09-16T00:50:57.748 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7[53103]: 2024-09-16T00:50:57.337+0000 7f310fb29640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.141392+0000 front 2024-09-16T00:50:24.141271+0000 (oldest deadline 2024-09-16T00:50:45.841218+0000) 2024-09-16T00:50:57.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:57.492+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:57.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:57.492+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:57.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:57.492+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:57.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:57.492+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:57.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:57 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:57.492+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:50:58.117 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:57 smithi081 podman[69662]: 2024-09-16 00:50:57.841623366 +0000 UTC m=+5.578475963 container died 7d38de5339edb8dc70b765a67d77c2221020a7a58745d0da44ba07e06a9558a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-16T00:50:58.125 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:57 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:57.680+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:58.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:57 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:57.680+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:58.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:57 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:57.680+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:58.484 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:58 smithi081 podman[69662]: 2024-09-16 00:50:58.208191407 +0000 UTC m=+5.945044032 container cleanup 7d38de5339edb8dc70b765a67d77c2221020a7a58745d0da44ba07e06a9558a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-8921c8e) 2024-09-16T00:50:58.485 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:58 smithi081 bash[69662]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7 2024-09-16T00:50:58.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:58 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:58.230+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:58.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:58 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:58.230+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:58.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:58 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:58.230+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:58.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:58 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:58.230+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.409990+0000 front 2024-09-16T00:50:32.409956+0000 (oldest deadline 2024-09-16T00:50:56.509910+0000) 2024-09-16T00:50:58.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:58 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:58.139+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:58.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:58 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:58.139+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:58.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:58 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:58.139+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:58.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:58 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:58.139+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:50:58.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:58 smithi081 podman[69677]: 2024-09-16 00:50:58.484342622 +0000 UTC m=+0.635312211 container remove 7d38de5339edb8dc70b765a67d77c2221020a7a58745d0da44ba07e06a9558a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T00:50:58.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:58.518+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:58.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:58.518+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:58.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:58.518+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:58.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:58.518+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:58.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:58 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:58.518+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:50:59.088 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:58 smithi081 podman[69763]: 2024-09-16 00:50:58.69673521 +0000 UTC m=+0.016221426 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:50:59.088 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:58 smithi081 podman[69763]: 2024-09-16 00:50:58.801981026 +0000 UTC m=+0.121467230 container create e68f806922c41f598cbc0fb39452ace52ef17b3b933246574ba629ba319cf0a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T00:50:59.088 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:59 smithi081 podman[69763]: 2024-09-16 00:50:59.08804187 +0000 UTC m=+0.407528072 container init e68f806922c41f598cbc0fb39452ace52ef17b3b933246574ba629ba319cf0a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T00:50:59.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:58 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:58.714+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:50:59.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:58 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:58.714+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:50:59.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:58 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:58.714+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:50:59.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:58 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:58.714+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:50:59.497 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:59 smithi081 podman[69763]: 2024-09-16 00:50:59.091300259 +0000 UTC m=+0.410786475 container start e68f806922c41f598cbc0fb39452ace52ef17b3b933246574ba629ba319cf0a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T00:50:59.497 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:59 smithi081 podman[69763]: 2024-09-16 00:50:59.188074642 +0000 UTC m=+0.507560846 container attach e68f806922c41f598cbc0fb39452ace52ef17b3b933246574ba629ba319cf0a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T00:50:59.498 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:50:59 smithi081 podman[69763]: 2024-09-16 00:50:59.273244084 +0000 UTC m=+0.592730298 container died e68f806922c41f598cbc0fb39452ace52ef17b3b933246574ba629ba319cf0a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-16T00:50:59.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:59 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:59.210+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:50:59.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:59 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:59.210+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:50:59.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:59 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:59.210+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:50:59.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:50:59 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:50:59.210+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.409990+0000 front 2024-09-16T00:50:32.409956+0000 (oldest deadline 2024-09-16T00:50:56.509910+0000) 2024-09-16T00:50:59.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:59 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:59.153+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:50:59.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:59 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:59.153+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:50:59.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:59 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:59.153+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:50:59.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:50:59 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:50:59.153+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:50:59.983 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:59.562+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:50:59.983 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:59.562+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:50:59.983 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:59.562+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:50:59.984 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:59.562+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:50:59.984 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:50:59 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:50:59.562+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:00.014 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T00:51:00.057 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:00.058 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T00:51:00.058 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-16T00:51:00.058 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.8 2024-09-16T00:51:00.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:59.688+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:00.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:59.688+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:00.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:59.688+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:00.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:50:59 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:50:59.688+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:00.372 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:00.189+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:00.372 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:00.189+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:00.372 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:00.189+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:00.372 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:00.189+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:00.372 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:00 smithi102 systemd[1]: Stopping Ceph osd.8 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:00.372 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:00.203+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:51:00.372 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:00.203+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:51:00.373 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:00.203+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:51:00.373 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:00.203+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.409990+0000 front 2024-09-16T00:50:32.409956+0000 (oldest deadline 2024-09-16T00:50:56.509910+0000) 2024-09-16T00:51:00.373 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:00.371+0000 7fb46e0f5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:00.373 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:00.371+0000 7fb46e0f5640 -1 osd.8 95 *** Got signal Terminated *** 2024-09-16T00:51:00.373 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:00 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:00.371+0000 7fb46e0f5640 -1 osd.8 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:51:00.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:00 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:00.516+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:00.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:00 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:00.516+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:00.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:00 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:00.516+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:00.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:00 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:00.516+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:00.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:00 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:00.516+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:01.027 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:00 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:00.699+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:01.027 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:00 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:00.699+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:01.027 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:00 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:00.699+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:01.028 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:00 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:00.699+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:01.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:01 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:01.223+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:01.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:01 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:01.223+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:01.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:01 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:01.223+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:01.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:01 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:01.223+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:01.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:01 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:01.206+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:51:01.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:01 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:01.206+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:51:01.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:01 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:01.206+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:51:01.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:01 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:01.206+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.409990+0000 front 2024-09-16T00:50:32.409956+0000 (oldest deadline 2024-09-16T00:50:56.509910+0000) 2024-09-16T00:51:01.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:01 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:01.550+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:01.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:01 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:01.550+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:01.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:01 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:01.550+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:01.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:01 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:01.550+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:01.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:01 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:01.550+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:02.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:01 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:01.742+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:02.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:01 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:01.742+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:02.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:01 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:01.742+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:02.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:01 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:01.742+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:02.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:02 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:02.213+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:51:02.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:02 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:02.213+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:51:02.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:02 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:02.213+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:51:02.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:02 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:02.213+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.409990+0000 front 2024-09-16T00:50:32.409956+0000 (oldest deadline 2024-09-16T00:50:56.509910+0000) 2024-09-16T00:51:02.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:02 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:02.186+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:02.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:02 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:02.186+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:02.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:02 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:02.186+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:02.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:02 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:02.186+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:02.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:02 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:02.541+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:02.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:02 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:02.541+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:02.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:02 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:02.541+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:02.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:02 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:02.541+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:02.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:02 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:02.541+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:03.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:02 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:02.750+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:03.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:02 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:02.750+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:03.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:02 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:02.750+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:03.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:02 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:02.750+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:03.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:03 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:03.193+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:03.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:03 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:03.193+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:03.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:03 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:03.193+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:03.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:03 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:03.193+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:03.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:03 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:03.189+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:51:03.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:03 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:03.189+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:51:03.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:03 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:03.189+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:51:03.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:03 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:03.189+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.409990+0000 front 2024-09-16T00:50:32.409956+0000 (oldest deadline 2024-09-16T00:50:56.509910+0000) 2024-09-16T00:51:03.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:03 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:03.189+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:36.510380+0000 front 2024-09-16T00:50:36.510426+0000 (oldest deadline 2024-09-16T00:51:02.410406+0000) 2024-09-16T00:51:03.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:03 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:03.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:03.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:03 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:03.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:03.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:03 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:03.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:03.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:03 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:03.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:03.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:03 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:03.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:04.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:03 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:03.717+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:04.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:03 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:03.717+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:04.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:03 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:03.717+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:04.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:03 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:03.717+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:04.220+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:04.220+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:04.220+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:04.220+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.409990+0000 front 2024-09-16T00:50:32.409956+0000 (oldest deadline 2024-09-16T00:50:56.509910+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:04.220+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:36.510380+0000 front 2024-09-16T00:50:36.510426+0000 (oldest deadline 2024-09-16T00:51:02.410406+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:04.209+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:04.209+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:04.209+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:04.209+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:04.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:04 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:04.209+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:04.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:04 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:04.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:04.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:04 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:04.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:04.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:04 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:04.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:04.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:04 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:04.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:04.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:04 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:04.577+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:05.129 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:04 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:04.670+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:05.129 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:04 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:04.670+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:05.129 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:04 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:04.670+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:05.129 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:04 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:04.670+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:05.129 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:04 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:04.670+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:40.505017+0000 front 2024-09-16T00:50:40.505042+0000 (oldest deadline 2024-09-16T00:51:04.005004+0000) 2024-09-16T00:51:05.549 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:05.271+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:04.707653+0000 front 2024-09-16T00:50:04.707500+0000 (oldest deadline 2024-09-16T00:50:34.708154+0000) 2024-09-16T00:51:05.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:05.271+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.708415+0000 front 2024-09-16T00:50:14.708569+0000 (oldest deadline 2024-09-16T00:50:40.608577+0000) 2024-09-16T00:51:05.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:05.271+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.609167+0000 front 2024-09-16T00:50:20.609131+0000 (oldest deadline 2024-09-16T00:50:46.509093+0000) 2024-09-16T00:51:05.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:05.271+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.409990+0000 front 2024-09-16T00:50:32.409956+0000 (oldest deadline 2024-09-16T00:50:56.509910+0000) 2024-09-16T00:51:05.550 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8[53320]: 2024-09-16T00:51:05.271+0000 7fb46a70b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:36.510380+0000 front 2024-09-16T00:50:36.510426+0000 (oldest deadline 2024-09-16T00:51:02.410406+0000) 2024-09-16T00:51:05.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:05.220+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:05.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:05.220+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:05.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:05.220+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:05.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:05.220+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:05.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:05 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:05.220+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:05.991 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:05 smithi102 podman[69799]: 2024-09-16 00:51:05.717284111 +0000 UTC m=+5.464496357 container died 7204089148a0412886fcfbb400e0304f62afc4783ddbd3c83926786de445face (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T00:51:05.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:05 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:05.572+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:05.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:05 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:05.572+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:05.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:05 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:05.572+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:05.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:05 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:05.572+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:05.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:05 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:05.572+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:05.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:05 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:05.572+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:06.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:05 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:05.665+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:06.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:05 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:05.665+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:06.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:05 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:05.665+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:06.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:05 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:05.665+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:06.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:05 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:05.665+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:40.505017+0000 front 2024-09-16T00:50:40.505042+0000 (oldest deadline 2024-09-16T00:51:04.005004+0000) 2024-09-16T00:51:06.299 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:06 smithi102 podman[69799]: 2024-09-16 00:51:06.088602246 +0000 UTC m=+5.835814491 container cleanup 7204089148a0412886fcfbb400e0304f62afc4783ddbd3c83926786de445face (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T00:51:06.300 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:06 smithi102 bash[69799]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8 2024-09-16T00:51:06.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:06 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:06.234+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:06.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:06 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:06.234+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:06.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:06 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:06.234+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:06.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:06 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:06.234+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:06.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:06 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:06.234+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:06.698 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:06 smithi102 podman[69814]: 2024-09-16 00:51:06.390086866 +0000 UTC m=+0.671793813 container remove 7204089148a0412886fcfbb400e0304f62afc4783ddbd3c83926786de445face (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T00:51:06.993 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:06 smithi102 podman[69905]: 2024-09-16 00:51:06.601350909 +0000 UTC m=+0.015867769 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:51:06.993 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:06 smithi102 podman[69905]: 2024-09-16 00:51:06.740713635 +0000 UTC m=+0.155230493 container create 2ca5a1941b00060605361f50b9622406e0029cafd22262f467096f57903765c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T00:51:06.993 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:06 smithi102 podman[69905]: 2024-09-16 00:51:06.993248761 +0000 UTC m=+0.407765615 container init 2ca5a1941b00060605361f50b9622406e0029cafd22262f467096f57903765c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:51:06.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:06 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:06.546+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:06.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:06 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:06.546+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:06.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:06 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:06.546+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:06.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:06 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:06.546+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:06.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:06 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:06.546+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:06.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:06 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:06.546+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:07.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:06 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:06.667+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:07.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:06 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:06.667+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:07.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:06 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:06.667+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:07.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:06 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:06.667+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:07.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:06 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:06.667+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:40.505017+0000 front 2024-09-16T00:50:40.505042+0000 (oldest deadline 2024-09-16T00:51:04.005004+0000) 2024-09-16T00:51:07.254 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:06 smithi102 podman[69905]: 2024-09-16 00:51:06.996646647 +0000 UTC m=+0.411163501 container start 2ca5a1941b00060605361f50b9622406e0029cafd22262f467096f57903765c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T00:51:07.254 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:07 smithi102 podman[69905]: 2024-09-16 00:51:07.132751117 +0000 UTC m=+0.547267980 container attach 2ca5a1941b00060605361f50b9622406e0029cafd22262f467096f57903765c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T00:51:07.254 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:51:07 smithi102 podman[69905]: 2024-09-16 00:51:07.183557638 +0000 UTC m=+0.598074493 container died 2ca5a1941b00060605361f50b9622406e0029cafd22262f467096f57903765c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-16T00:51:07.538 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:07 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:07.253+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:07.539 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:07 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:07.253+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:07.539 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:07 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:07.253+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:07.539 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:07 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:07.253+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:07.539 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:07 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:07.253+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:07.859 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.8.service' 2024-09-16T00:51:07.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:07.897 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-16T00:51:07.897 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-16T00:51:07.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.9 2024-09-16T00:51:07.927 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:07 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:07.642+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:07.927 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:07 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:07.642+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:07.928 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:07 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:07.642+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:07.928 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:07 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:07.642+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:07.928 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:07 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:07.642+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:40.505017+0000 front 2024-09-16T00:50:40.505042+0000 (oldest deadline 2024-09-16T00:51:04.005004+0000) 2024-09-16T00:51:07.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:07 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:07.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:07.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:07 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:07.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:07.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:07 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:07.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:07.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:07 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:07.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:07.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:07 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:07.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:07.999 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:07 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:07.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:08.201 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:07 smithi031 systemd[1]: Stopping Ceph osd.9 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:08.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:08 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:08.233+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:08.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:08 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:08.233+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:08.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:08 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:08.233+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:08.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:08 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:08.233+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:08.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:08 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:08.233+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:08.625 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:08.197+0000 7f3f33590640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:08.626 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:08.197+0000 7f3f33590640 -1 osd.9 95 *** Got signal Terminated *** 2024-09-16T00:51:08.626 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:08.197+0000 7f3f33590640 -1 osd.9 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:51:08.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:08 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:08.472+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:08.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:08 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:08.472+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:08.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:08 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:08.472+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:08.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:08 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:08.472+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:08.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:08 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:08.472+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:08.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:08 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:08.472+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:09.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:08.684+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:09.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:08.684+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:09.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:08.684+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:09.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:08.684+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:09.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:08 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:08.684+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:40.505017+0000 front 2024-09-16T00:50:40.505042+0000 (oldest deadline 2024-09-16T00:51:04.005004+0000) 2024-09-16T00:51:09.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:09 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:09.264+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:09.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:09 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:09.264+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:09.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:09 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:09.264+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:09.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:09 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:09.264+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:09.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:09 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:09.264+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:09.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:09 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:09.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:09.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:09 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:09.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:09.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:09 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:09.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:09.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:09 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:09.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:09.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:09 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:09.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:09.999 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:09 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:09.511+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:10.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:09 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:09.717+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:10.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:09 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:09.717+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:10.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:09 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:09.717+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:10.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:09 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:09.717+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:10.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:09 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:09.717+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:40.505017+0000 front 2024-09-16T00:50:40.505042+0000 (oldest deadline 2024-09-16T00:51:04.005004+0000) 2024-09-16T00:51:10.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:10 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:10.256+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:10.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:10 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:10.256+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:10.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:10 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:10.256+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:10.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:10 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:10.256+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:10.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:10 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:10.256+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:10.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:10 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:10.551+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:10.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:10 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:10.551+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:10.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:10 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:10.551+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:10.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:10 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:10.551+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:10.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:10 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:10.551+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:10.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:10 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:10.551+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:10.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:10 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:10.551+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:11.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:10 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:10.739+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:11.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:10 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:10.739+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:11.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:10 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:10.739+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:11.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:10 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:10.739+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:11.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:10 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:10.739+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:40.505017+0000 front 2024-09-16T00:50:40.505042+0000 (oldest deadline 2024-09-16T00:51:04.005004+0000) 2024-09-16T00:51:11.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:11 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:11.225+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:11.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:11 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:11.225+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:11.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:11 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:11.225+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:11.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:11 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:11.225+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:11.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:11 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:11.225+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:11.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:11 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:11.225+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:11.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:11 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:11.513+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:11.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:11 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:11.513+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:11.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:11 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:11.513+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:11.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:11 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:11.513+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:11.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:11 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:11.513+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:11.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:11 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:11.513+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:11.998 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:11 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:11.513+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:12.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:11 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:11.761+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:12.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:11 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:11.761+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:12.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:11 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:11.761+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:12.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:11 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:11.761+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:12.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:11 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:11.761+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:40.505017+0000 front 2024-09-16T00:50:40.505042+0000 (oldest deadline 2024-09-16T00:51:04.005004+0000) 2024-09-16T00:51:12.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:12 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:12.231+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:12.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:12 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:12.231+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:12.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:12 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:12.231+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:12.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:12 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:12.231+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:12.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:12 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:12.231+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:12.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:12 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:12.231+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:12.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:12 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:12.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:12.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:12 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:12.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:12.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:12 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:12.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:12.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:12 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:12.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:12.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:12 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:12.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:12.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:12 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:12.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:12.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:12 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:12.475+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:13.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:12.788+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.996962+0000 front 2024-09-16T00:50:06.997015+0000 (oldest deadline 2024-09-16T00:50:29.797774+0000) 2024-09-16T00:51:13.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:12.788+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.500534+0000 front 2024-09-16T00:50:15.500567+0000 (oldest deadline 2024-09-16T00:50:38.400701+0000) 2024-09-16T00:51:13.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:12.788+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.701701+0000 front 2024-09-16T00:50:20.701747+0000 (oldest deadline 2024-09-16T00:50:46.001847+0000) 2024-09-16T00:51:13.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:12.788+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:32.903987+0000 front 2024-09-16T00:50:32.903969+0000 (oldest deadline 2024-09-16T00:50:58.204110+0000) 2024-09-16T00:51:13.126 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:12 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9[62592]: 2024-09-16T00:51:12.788+0000 7f3f2bb9e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:40.505017+0000 front 2024-09-16T00:50:40.505042+0000 (oldest deadline 2024-09-16T00:51:04.005004+0000) 2024-09-16T00:51:13.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:13 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:13.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:13.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:13 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:13.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:13.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:13 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:13.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:13.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:13 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:13.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:13.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:13 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:13.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:13.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:13 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:13.227+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:13.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:13 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:13.451+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:13.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:13 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:13.451+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:13.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:13 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:13.451+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:13.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:13 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:13.451+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:13.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:13 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:13.451+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:13.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:13 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:13.451+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:13.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:13 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:13.451+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:14.032 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:13 smithi031 podman[100735]: 2024-09-16 00:51:13.739924152 +0000 UTC m=+5.639451604 container died 2a909b49bd19e09d516934e387b0c4e058582ff3b4ed137f4762e8070907010c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:51:14.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:14 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:14.194+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:14.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:14 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:14.194+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:14.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:14 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:14.194+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:14.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:14 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:14.194+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:14.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:14 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:14.194+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:14.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:14 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:14.194+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:14.573 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:14 smithi031 podman[100735]: 2024-09-16 00:51:14.321976581 +0000 UTC m=+6.221504033 container cleanup 2a909b49bd19e09d516934e387b0c4e058582ff3b4ed137f4762e8070907010c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:51:14.573 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:14 smithi031 bash[100735]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9 2024-09-16T00:51:14.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:14 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:14.402+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:14.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:14 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:14.402+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:14.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:14 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:14.402+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:14.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:14 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:14.402+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:14.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:14 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:14.402+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:14.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:14 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:14.402+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:14.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:14 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:14.402+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:14.875 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:14 smithi031 podman[100753]: 2024-09-16 00:51:14.57341082 +0000 UTC m=+0.825430978 container remove 2a909b49bd19e09d516934e387b0c4e058582ff3b4ed137f4762e8070907010c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T00:51:15.243 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:14 smithi031 podman[100845]: 2024-09-16 00:51:14.847302383 +0000 UTC m=+0.016070988 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:51:15.243 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:14 smithi031 podman[100845]: 2024-09-16 00:51:14.966214128 +0000 UTC m=+0.134982759 container create 8db2e770124ad3be12e9abc9ff6238df92d96789c1b7441388013926574a5303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:51:15.243 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:15 smithi031 podman[100845]: 2024-09-16 00:51:15.15201937 +0000 UTC m=+0.320787972 container init 8db2e770124ad3be12e9abc9ff6238df92d96789c1b7441388013926574a5303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T00:51:15.243 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:15 smithi031 podman[100845]: 2024-09-16 00:51:15.155837906 +0000 UTC m=+0.324606503 container start 8db2e770124ad3be12e9abc9ff6238df92d96789c1b7441388013926574a5303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:51:15.243 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:15 smithi031 podman[100845]: 2024-09-16 00:51:15.243233411 +0000 UTC m=+0.412001999 container attach 8db2e770124ad3be12e9abc9ff6238df92d96789c1b7441388013926574a5303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T00:51:15.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:15 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:15.161+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:15.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:15 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:15.161+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:15.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:15 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:15.161+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:15.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:15 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:15.161+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:15.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:15 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:15.161+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:15.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:15 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:15.161+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:15.626 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:51:15 smithi031 podman[100845]: 2024-09-16 00:51:15.349236531 +0000 UTC m=+0.518005124 container died 8db2e770124ad3be12e9abc9ff6238df92d96789c1b7441388013926574a5303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-16T00:51:15.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:15 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:15.375+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:15.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:15 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:15.375+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:15.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:15 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:15.375+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:15.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:15 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:15.375+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:15.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:15 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:15.375+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:15.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:15 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:15.375+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:15.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:15 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:15.375+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:16.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.9.service' 2024-09-16T00:51:16.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:16.257 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-16T00:51:16.257 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-16T00:51:16.257 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.10 2024-09-16T00:51:16.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:16 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:16.113+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:16.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:16 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:16.113+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:16.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:16 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:16.113+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:16.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:16 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:16.113+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:16.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:16 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:16.113+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:16.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:16 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:16.113+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 systemd[1]: Stopping Ceph osd.10 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.326+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.326+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.326+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.326+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.326+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.326+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.326+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.576+0000 7fc51a88f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.576+0000 7fc51a88f640 -1 osd.10 95 *** Got signal Terminated *** 2024-09-16T00:51:16.579 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:16 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:16.576+0000 7fc51a88f640 -1 osd.10 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:51:17.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:17 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:17.146+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:17.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:17 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:17.146+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:17.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:17 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:17.146+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:17.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:17 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:17.146+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:17.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:17 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:17.146+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:17.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:17 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:17.146+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:17.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:17 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:17.349+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:17.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:17 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:17.349+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:17.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:17 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:17.349+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:17.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:17 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:17.349+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:17.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:17 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:17.349+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:17.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:17 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:17.349+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:17.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:17 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:17.349+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:18.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:18 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:18.150+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:18.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:18 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:18.150+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:18.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:18 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:18.150+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:18.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:18 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:18.150+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:18.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:18 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:18.150+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:18.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:18 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:18.150+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:18.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:18 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:18.355+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:18.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:18 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:18.355+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:18.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:18 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:18.355+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:18.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:18 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:18.355+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:18.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:18 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:18.355+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:18.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:18 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:18.355+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:18.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:18 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:18.355+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:19.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:19 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:19.134+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:19.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:19 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:19.134+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:19.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:19 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:19.134+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:19.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:19 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:19.134+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:19.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:19 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:19.134+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:19.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:19 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:19.134+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:19.700 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:19 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:19.330+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:19.700 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:19 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:19.330+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:19.701 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:19 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:19.330+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:19.701 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:19 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:19.330+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:19.701 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:19 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:19.330+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:19.701 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:19 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:19.330+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:19.701 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:19 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:19.330+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:20.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:20 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:20.095+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:20.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:20 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:20.095+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:20.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:20 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:20.095+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:20.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:20 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:20.095+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:20.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:20 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:20.095+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:20.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:20 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:20.095+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:20.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:20 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:20.095+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:20.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:20 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:20.376+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:20.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:20 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:20.376+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:20.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:20 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:20.376+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:20.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:20 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:20.376+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:20.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:20 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:20.376+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:20.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:20 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:20.376+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:20.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:20 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:20.376+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:21.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:21 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:21.106+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:21.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:21 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:21.106+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:21.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:21 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:21.106+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:21.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:21 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:21.106+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:21.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:21 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:21.106+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:21.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:21 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:21.106+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:21.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:21 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:21.106+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:21.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:21 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:21.407+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:50:00.040170+0000 front 2024-09-16T00:50:00.040019+0000 (oldest deadline 2024-09-16T00:50:25.340077+0000) 2024-09-16T00:51:21.747 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:21 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:21.407+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:50:06.440967+0000 front 2024-09-16T00:50:06.440991+0000 (oldest deadline 2024-09-16T00:50:29.941175+0000) 2024-09-16T00:51:21.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:21 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:21.407+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:14.042297+0000 front 2024-09-16T00:50:14.042315+0000 (oldest deadline 2024-09-16T00:50:38.142203+0000) 2024-09-16T00:51:21.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:21 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:21.407+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:24.543627+0000 front 2024-09-16T00:50:24.543756+0000 (oldest deadline 2024-09-16T00:50:47.443632+0000) 2024-09-16T00:51:21.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:21 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:21.407+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:31.544690+0000 front 2024-09-16T00:50:31.544717+0000 (oldest deadline 2024-09-16T00:50:53.844903+0000) 2024-09-16T00:51:21.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:21 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:21.407+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.746077+0000 front 2024-09-16T00:50:39.746159+0000 (oldest deadline 2024-09-16T00:51:04.946241+0000) 2024-09-16T00:51:21.748 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:21 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10[60068]: 2024-09-16T00:51:21.407+0000 7fc512e9d640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:48.947690+0000 front 2024-09-16T00:50:48.947744+0000 (oldest deadline 2024-09-16T00:51:10.047693+0000) 2024-09-16T00:51:22.204 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:21 smithi081 podman[69879]: 2024-09-16 00:51:21.918424229 +0000 UTC m=+5.468289706 container died 315eb81b5a3c89385eee314912e0fff0db471508141d9a2c985462b4570b2d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-16T00:51:22.498 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:22 smithi081 podman[69879]: 2024-09-16 00:51:22.302979845 +0000 UTC m=+5.852845321 container cleanup 315eb81b5a3c89385eee314912e0fff0db471508141d9a2c985462b4570b2d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T00:51:22.498 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:22 smithi081 bash[69879]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10 2024-09-16T00:51:22.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:22 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:22.118+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:22.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:22 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:22.118+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:22.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:22 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:22.118+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:22.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:22 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:22.118+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:22.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:22 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:22.118+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:22.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:22 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:22.118+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:22.551 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:22 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:22.118+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:22.896 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:22 smithi081 podman[69897]: 2024-09-16 00:51:22.587580281 +0000 UTC m=+0.659997840 container remove 315eb81b5a3c89385eee314912e0fff0db471508141d9a2c985462b4570b2d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:51:23.216 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:22 smithi081 podman[69986]: 2024-09-16 00:51:22.896584066 +0000 UTC m=+0.111853133 container create 55ac5d1ea5c1dcbe8844ce541c946f4719d3b60e2a945f2978d813f723d3eb50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T00:51:23.216 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:22 smithi081 podman[69986]: 2024-09-16 00:51:22.800301611 +0000 UTC m=+0.015570691 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:51:23.216 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:23 smithi081 podman[69986]: 2024-09-16 00:51:23.132596738 +0000 UTC m=+0.347865812 container init 55ac5d1ea5c1dcbe8844ce541c946f4719d3b60e2a945f2978d813f723d3eb50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T00:51:23.216 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:23 smithi081 podman[69986]: 2024-09-16 00:51:23.135875653 +0000 UTC m=+0.351144728 container start 55ac5d1ea5c1dcbe8844ce541c946f4719d3b60e2a945f2978d813f723d3eb50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T00:51:23.216 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:23 smithi081 podman[69986]: 2024-09-16 00:51:23.215877627 +0000 UTC m=+0.431146701 container attach 55ac5d1ea5c1dcbe8844ce541c946f4719d3b60e2a945f2978d813f723d3eb50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-16T00:51:23.497 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:51:23 smithi081 podman[69986]: 2024-09-16 00:51:23.320675823 +0000 UTC m=+0.535944902 container died 55ac5d1ea5c1dcbe8844ce541c946f4719d3b60e2a945f2978d813f723d3eb50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:51:23.552 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:23 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:23.094+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:23.553 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:23 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:23.094+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:23.553 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:23 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:23.094+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:23.553 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:23 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:23.094+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:23.553 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:23 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:23.094+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:23.553 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:23 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:23.094+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:23.553 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:23 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:23.094+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:23.870 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.10.service' 2024-09-16T00:51:23.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:23.911 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-16T00:51:23.911 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-16T00:51:23.911 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.11 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:23 smithi102 systemd[1]: Stopping Ceph osd.11 for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.096+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.096+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.096+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.096+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.096+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.096+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.096+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.232+0000 7f2e62ce9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:24.234 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.232+0000 7f2e62ce9640 -1 osd.11 95 *** Got signal Terminated *** 2024-09-16T00:51:24.235 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:24 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:24.232+0000 7f2e62ce9640 -1 osd.11 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:51:25.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:25.059+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:25.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:25.059+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:25.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:25.059+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:25.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:25.059+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:25.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:25.059+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:25.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:25.059+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:25.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:25 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:25.059+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:26.549 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:26 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:26.053+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:26.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:26 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:26.053+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:26.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:26 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:26.053+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:26.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:26 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:26.053+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:26.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:26 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:26.053+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:26.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:26 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:26.053+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:26.550 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:26 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:26.053+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:27.501 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:27.049+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:27.502 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:27.049+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:27.502 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:27.049+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:27.502 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:27.049+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:27.502 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:27.049+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:27.502 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:27.049+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:27.502 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:27 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:27.049+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:28.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:28.038+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:28.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:28.038+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:28.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:28.038+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:28.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:28.038+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:28.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:28.038+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:28.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:28.038+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:28.301 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:28.038+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:28.301 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:28 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:28.038+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6820 osd.8 since back 2024-09-16T00:50:59.671971+0000 front 2024-09-16T00:50:59.672052+0000 (oldest deadline 2024-09-16T00:51:27.272559+0000) 2024-09-16T00:51:29.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:29.047+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:49:59.062642+0000 front 2024-09-16T00:49:59.062657+0000 (oldest deadline 2024-09-16T00:50:23.162595+0000) 2024-09-16T00:51:29.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:29.047+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:50:15.965078+0000 front 2024-09-16T00:50:15.965176+0000 (oldest deadline 2024-09-16T00:50:40.665300+0000) 2024-09-16T00:51:29.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:29.047+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:50:20.665841+0000 front 2024-09-16T00:50:20.665781+0000 (oldest deadline 2024-09-16T00:50:46.565839+0000) 2024-09-16T00:51:29.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:29.047+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:50:29.467144+0000 front 2024-09-16T00:50:29.467054+0000 (oldest deadline 2024-09-16T00:50:53.566954+0000) 2024-09-16T00:51:29.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:29.047+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:50:39.368155+0000 front 2024-09-16T00:50:39.368200+0000 (oldest deadline 2024-09-16T00:51:03.968884+0000) 2024-09-16T00:51:29.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:29.047+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:50:47.971093+0000 front 2024-09-16T00:50:47.970928+0000 (oldest deadline 2024-09-16T00:51:10.870682+0000) 2024-09-16T00:51:29.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:29.047+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:50:55.571759+0000 front 2024-09-16T00:50:55.571820+0000 (oldest deadline 2024-09-16T00:51:19.671646+0000) 2024-09-16T00:51:29.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:29 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11[60026]: 2024-09-16T00:51:29.047+0000 7f2e5b2f7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.102:6820 osd.8 since back 2024-09-16T00:50:59.671971+0000 front 2024-09-16T00:50:59.672052+0000 (oldest deadline 2024-09-16T00:51:27.272559+0000) 2024-09-16T00:51:29.992 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:29 smithi102 podman[70019]: 2024-09-16 00:51:29.70734552 +0000 UTC m=+5.603537304 container died 5332fb60da3463379361bd265ad37d26ffeaad4ae082bc9d96cb1d068e05cd75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T00:51:30.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:30 smithi102 podman[70019]: 2024-09-16 00:51:30.091722794 +0000 UTC m=+5.987914575 container cleanup 5332fb60da3463379361bd265ad37d26ffeaad4ae082bc9d96cb1d068e05cd75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-16T00:51:30.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:30 smithi102 bash[70019]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11 2024-09-16T00:51:30.705 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:30 smithi102 podman[70034]: 2024-09-16 00:51:30.35120048 +0000 UTC m=+0.635612335 container remove 5332fb60da3463379361bd265ad37d26ffeaad4ae082bc9d96cb1d068e05cd75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:51:31.004 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:30 smithi102 podman[70124]: 2024-09-16 00:51:30.608007329 +0000 UTC m=+0.015921762 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:51:31.004 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:30 smithi102 podman[70124]: 2024-09-16 00:51:30.710160464 +0000 UTC m=+0.118074896 container create f6e9d5e8a0adecd7686ea70712413bc771957085d7504607be162c83dd11f3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:51:31.004 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:30 smithi102 podman[70124]: 2024-09-16 00:51:30.920902845 +0000 UTC m=+0.328817277 container init f6e9d5e8a0adecd7686ea70712413bc771957085d7504607be162c83dd11f3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:51:31.004 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:30 smithi102 podman[70124]: 2024-09-16 00:51:30.924097306 +0000 UTC m=+0.332011749 container start f6e9d5e8a0adecd7686ea70712413bc771957085d7504607be162c83dd11f3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:51:31.004 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:31 smithi102 podman[70124]: 2024-09-16 00:51:31.00421344 +0000 UTC m=+0.412127884 container attach f6e9d5e8a0adecd7686ea70712413bc771957085d7504607be162c83dd11f3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:51:31.300 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:51:31 smithi102 podman[70147]: 2024-09-16 00:51:31.140324166 +0000 UTC m=+0.016542402 container died f6e9d5e8a0adecd7686ea70712413bc771957085d7504607be162c83dd11f3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T00:51:31.745 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@osd.11.service' 2024-09-16T00:51:31.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:31.783 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-16T00:51:31.783 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-16T00:51:31.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.a 2024-09-16T00:51:32.075 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:51:31 smithi031 systemd[1]: Stopping Ceph mds.a for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:32.341 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:51:32 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-a[69455]: 2024-09-16T00:51:32.074+0000 7f73fcaf7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:32.341 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:51:32 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-a[69455]: 2024-09-16T00:51:32.074+0000 7f73fcaf7640 -1 mds.a *** got signal Terminated *** 2024-09-16T00:51:36.376 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:51:36 smithi031 podman[100961]: 2024-09-16 00:51:36.085332602 +0000 UTC m=+4.096800167 container died ce1c6b7e983dac015b8458a3d042d838610475f2c10428b4e6bc4f53a4602caf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:51:36.716 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:51:36 smithi031 podman[100961]: 2024-09-16 00:51:36.440154517 +0000 UTC m=+4.451622123 container cleanup ce1c6b7e983dac015b8458a3d042d838610475f2c10428b4e6bc4f53a4602caf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-a, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:51:36.716 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:51:36 smithi031 bash[100961]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-a 2024-09-16T00:51:36.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.a.service' 2024-09-16T00:51:36.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:36.925 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-16T00:51:36.925 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-16T00:51:36.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.d 2024-09-16T00:51:37.253 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:51:37 smithi031 systemd[1]: Stopping Ceph mds.d for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:37.626 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:51:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-d[68277]: 2024-09-16T00:51:37.252+0000 7fa2fd870640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:37.626 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:51:37 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-d[68277]: 2024-09-16T00:51:37.252+0000 7fa2fd870640 -1 mds.d *** got signal Terminated *** 2024-09-16T00:51:40.819 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:51:40 smithi031 podman[101096]: 2024-09-16 00:51:40.521057434 +0000 UTC m=+3.374240800 container died 71e2abc1939901d9ab872feda504ad59c65a480e79dd98700a9484f54bf406e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:51:41.125 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:51:40 smithi031 podman[101096]: 2024-09-16 00:51:40.90185954 +0000 UTC m=+3.755042903 container cleanup 71e2abc1939901d9ab872feda504ad59c65a480e79dd98700a9484f54bf406e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-16T00:51:41.125 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:51:40 smithi031 bash[101096]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-d 2024-09-16T00:51:41.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.d.service' 2024-09-16T00:51:41.297 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:41.298 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-16T00:51:41.298 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-16T00:51:41.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.g 2024-09-16T00:51:41.876 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:51:41 smithi031 systemd[1]: Stopping Ceph mds.g for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:41.876 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:51:41 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-g[69061]: 2024-09-16T00:51:41.624+0000 7fb839cd0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:41.876 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:51:41 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-g[69061]: 2024-09-16T00:51:41.624+0000 7fb839cd0640 -1 mds.g *** got signal Terminated *** 2024-09-16T00:51:45.376 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:51:45 smithi031 podman[101218]: 2024-09-16 00:51:45.10777313 +0000 UTC m=+3.589098526 container died c8e9d3c8e852688957255669f9b12c6203497a69f2565c3749edd582b2050475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True) 2024-09-16T00:51:45.725 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:51:45 smithi031 podman[101218]: 2024-09-16 00:51:45.465958246 +0000 UTC m=+3.947283641 container cleanup c8e9d3c8e852688957255669f9b12c6203497a69f2565c3749edd582b2050475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-g, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T00:51:45.725 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:51:45 smithi031 bash[101218]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-g 2024-09-16T00:51:45.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.g.service' 2024-09-16T00:51:45.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:45.944 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-16T00:51:45.944 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-16T00:51:45.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.j 2024-09-16T00:51:46.278 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:51:46 smithi031 systemd[1]: Stopping Ceph mds.j for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:46.626 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:51:46 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-j[67864]: 2024-09-16T00:51:46.277+0000 7fcbded12640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:46.626 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:51:46 smithi031 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-j[67864]: 2024-09-16T00:51:46.277+0000 7fcbded12640 -1 mds.j *** got signal Terminated *** 2024-09-16T00:51:49.504 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:51:49 smithi031 podman[101342]: 2024-09-16 00:51:49.225571563 +0000 UTC m=+3.059441473 container died f856e32d6506faf00db870a064f49ebb51b13fb890a00f26579e1e762f274c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T00:51:49.876 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:51:49 smithi031 podman[101342]: 2024-09-16 00:51:49.644329326 +0000 UTC m=+3.478199276 container cleanup f856e32d6506faf00db870a064f49ebb51b13fb890a00f26579e1e762f274c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T00:51:49.876 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:51:49 smithi031 bash[101342]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-j 2024-09-16T00:51:50.134 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:51:49 smithi031 podman[101355]: 2024-09-16 00:51:49.884286878 +0000 UTC m=+0.655219677 container remove f856e32d6506faf00db870a064f49ebb51b13fb890a00f26579e1e762f274c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True) 2024-09-16T00:51:50.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.j.service' 2024-09-16T00:51:50.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:50.171 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-16T00:51:50.172 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-16T00:51:50.172 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.b 2024-09-16T00:51:50.462 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:51:50 smithi081 systemd[1]: Stopping Ceph mds.b for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:50.747 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:51:50 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b[65567]: 2024-09-16T00:51:50.461+0000 7f148768b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:50.747 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:51:50 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b[65567]: 2024-09-16T00:51:50.461+0000 7f148768b640 -1 mds.b *** got signal Terminated *** 2024-09-16T00:51:54.336 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:51:54 smithi081 podman[70088]: 2024-09-16 00:51:54.009195248 +0000 UTC m=+3.643796829 container died 00e3bc8e521c12196171330c3e7cc1fc03235a1edf0b684cf9cf0694051eccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:51:54.336 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:51:54 smithi081 podman[70088]: 2024-09-16 00:51:54.336143119 +0000 UTC m=+3.970744689 container cleanup 00e3bc8e521c12196171330c3e7cc1fc03235a1edf0b684cf9cf0694051eccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:51:54.747 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:51:54 smithi081 bash[70088]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b 2024-09-16T00:51:54.747 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:51:54 smithi081 podman[70113]: 2024-09-16 00:51:54.55423574 +0000 UTC m=+0.540012691 container remove 00e3bc8e521c12196171330c3e7cc1fc03235a1edf0b684cf9cf0694051eccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e) 2024-09-16T00:51:54.769 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.b.service' 2024-09-16T00:51:54.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:51:54.806 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-16T00:51:54.806 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-16T00:51:54.806 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.e 2024-09-16T00:51:55.149 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:51:54 smithi081 systemd[1]: Stopping Ceph mds.e for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:51:55.501 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:51:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e[64957]: 2024-09-16T00:51:55.148+0000 7f7519a74640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:51:55.501 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:51:55 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e[64957]: 2024-09-16T00:51:55.148+0000 7f7519a74640 -1 mds.e *** got signal Terminated *** 2024-09-16T00:51:59.474 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:51:59 smithi081 podman[70222]: 2024-09-16 00:51:59.134694049 +0000 UTC m=+4.100146888 container died df6645692f3317be96e7f1e9d277a3340ec199207fc122ee84f9936c186bfe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T00:51:59.747 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:51:59 smithi081 podman[70222]: 2024-09-16 00:51:59.573487545 +0000 UTC m=+4.538940394 container cleanup df6645692f3317be96e7f1e9d277a3340ec199207fc122ee84f9936c186bfe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:51:59.747 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:51:59 smithi081 bash[70222]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e 2024-09-16T00:52:00.049 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.e.service' 2024-09-16T00:52:00.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:51:59 smithi081 podman[70234]: 2024-09-16 00:51:59.800345536 +0000 UTC m=+0.662684382 container remove df6645692f3317be96e7f1e9d277a3340ec199207fc122ee84f9936c186bfe77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:52:00.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:52:00 smithi081 systemd[1]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.e.service: Deactivated successfully. 2024-09-16T00:52:00.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:52:00 smithi081 systemd[1]: Stopped Ceph mds.e for ccb601ea-73c2-11ef-bceb-c7b262605968. 2024-09-16T00:52:00.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:52:00 smithi081 systemd[1]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.e.service: Consumed 1.004s CPU time. 2024-09-16T00:52:00.084 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:52:00.084 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-16T00:52:00.084 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-16T00:52:00.084 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.h 2024-09-16T00:52:00.402 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:52:00 smithi081 systemd[1]: Stopping Ceph mds.h for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:52:00.747 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:52:00 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-h[65256]: 2024-09-16T00:52:00.401+0000 7f0fa8528640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:52:00.747 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:52:00 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-h[65256]: 2024-09-16T00:52:00.401+0000 7f0fa8528640 -1 mds.h *** got signal Terminated *** 2024-09-16T00:52:03.368 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:52:03 smithi081 podman[70340]: 2024-09-16 00:52:03.04336833 +0000 UTC m=+2.744768464 container died 3a4d39797acbb954342f10bd17310c5bb3c025c9351a8f565785a8495a31474b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True) 2024-09-16T00:52:03.747 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:52:03 smithi081 podman[70340]: 2024-09-16 00:52:03.368748077 +0000 UTC m=+3.070148198 container cleanup 3a4d39797acbb954342f10bd17310c5bb3c025c9351a8f565785a8495a31474b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:52:03.747 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:52:03 smithi081 bash[70340]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-h 2024-09-16T00:52:03.747 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:52:03 smithi081 podman[70353]: 2024-09-16 00:52:03.570029562 +0000 UTC m=+0.521871785 container remove 3a4d39797acbb954342f10bd17310c5bb3c025c9351a8f565785a8495a31474b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-16T00:52:03.798 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.h.service' 2024-09-16T00:52:03.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:52:03.831 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-16T00:52:03.831 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-16T00:52:03.831 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.k 2024-09-16T00:52:04.139 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:52:03 smithi081 systemd[1]: Stopping Ceph mds.k for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:52:04.497 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:52:04 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-k[64646]: 2024-09-16T00:52:04.138+0000 7f523f475640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:52:04.497 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:52:04 smithi081 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-k[64646]: 2024-09-16T00:52:04.138+0000 7f523f475640 -1 mds.k *** got signal Terminated *** 2024-09-16T00:52:05.906 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:52:05 smithi081 podman[70461]: 2024-09-16 00:52:05.613933113 +0000 UTC m=+1.568277158 container died 616001c7e847b63845c02613cc7d84c9f4b5bd4e46968c91698c2154ae5016d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-16T00:52:06.247 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:52:05 smithi081 podman[70461]: 2024-09-16 00:52:05.997702262 +0000 UTC m=+1.952046315 container cleanup 616001c7e847b63845c02613cc7d84c9f4b5bd4e46968c91698c2154ae5016d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-k, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T00:52:06.247 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:52:05 smithi081 bash[70461]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-k 2024-09-16T00:52:06.432 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.k.service' 2024-09-16T00:52:06.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:52:06.464 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-16T00:52:06.465 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-16T00:52:06.465 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.c 2024-09-16T00:52:06.759 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:52:06 smithi102 systemd[1]: Stopping Ceph mds.c for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:52:07.009 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:52:06 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-c[65005]: 2024-09-16T00:52:06.756+0000 7f93b576f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:52:07.009 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:52:06 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-c[65005]: 2024-09-16T00:52:06.756+0000 7f93b576f640 -1 mds.c *** got signal Terminated *** 2024-09-16T00:52:08.633 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:52:08 smithi102 podman[70230]: 2024-09-16 00:52:08.366198046 +0000 UTC m=+1.713484956 container died 95a1cc22f1c2628f4485f174f3a84fe7b07d5d84acdc0c3cbe05d6acbe2edf0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T00:52:08.992 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:52:08 smithi102 podman[70230]: 2024-09-16 00:52:08.72451698 +0000 UTC m=+2.071803932 container cleanup 95a1cc22f1c2628f4485f174f3a84fe7b07d5d84acdc0c3cbe05d6acbe2edf0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T00:52:08.992 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:52:08 smithi102 bash[70230]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-c 2024-09-16T00:52:09.142 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.c.service' 2024-09-16T00:52:09.178 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:52:09.178 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-16T00:52:09.178 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-16T00:52:09.178 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.f 2024-09-16T00:52:09.495 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:52:09 smithi102 systemd[1]: Stopping Ceph mds.f for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:52:09.799 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:52:09 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-f[64705]: 2024-09-16T00:52:09.494+0000 7fa80a9e6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:52:09.800 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:52:09 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-f[64705]: 2024-09-16T00:52:09.494+0000 7fa80a9e6640 -1 mds.f *** got signal Terminated *** 2024-09-16T00:52:13.779 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:52:13 smithi102 podman[70362]: 2024-09-16 00:52:13.507051797 +0000 UTC m=+4.114360810 container died 99fe3278e5e6ed360037b61bc3847aafec1d3f685c7a2480df7e24cc459f6df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:52:14.049 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:52:13 smithi102 podman[70362]: 2024-09-16 00:52:13.886220611 +0000 UTC m=+4.493529626 container cleanup 99fe3278e5e6ed360037b61bc3847aafec1d3f685c7a2480df7e24cc459f6df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:52:14.050 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:52:13 smithi102 bash[70362]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-f 2024-09-16T00:52:14.316 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.f.service' 2024-09-16T00:52:14.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:52:14.349 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-16T00:52:14.350 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-16T00:52:14.350 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.i 2024-09-16T00:52:14.674 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:52:14 smithi102 systemd[1]: Stopping Ceph mds.i for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:52:15.049 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:52:14 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i[65307]: 2024-09-16T00:52:14.673+0000 7f8b33a57640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:52:15.049 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:52:14 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i[65307]: 2024-09-16T00:52:14.673+0000 7f8b33a57640 -1 mds.i *** got signal Terminated *** 2024-09-16T00:52:18.299 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:52:18 smithi102 podman[70483]: 2024-09-16 00:52:18.001314935 +0000 UTC m=+3.433803241 container died 76b0ed73d828e6c29cc60354366772d4bf017aac4281287699ed3da340849e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T00:52:18.641 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:52:18 smithi102 podman[70483]: 2024-09-16 00:52:18.406638133 +0000 UTC m=+3.839126438 container cleanup 76b0ed73d828e6c29cc60354366772d4bf017aac4281287699ed3da340849e8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T00:52:18.641 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:52:18 smithi102 bash[70483]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-i 2024-09-16T00:52:18.775 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.i.service' 2024-09-16T00:52:18.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:52:18.808 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-16T00:52:18.808 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-16T00:52:18.809 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.l 2024-09-16T00:52:19.049 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:52:18 smithi102 systemd[1]: Stopping Ceph mds.l for ccb601ea-73c2-11ef-bceb-c7b262605968... 2024-09-16T00:52:19.549 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:52:19 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l[64401]: 2024-09-16T00:52:19.151+0000 7f4cfb95f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:52:19.549 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:52:19 smithi102 ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l[64401]: 2024-09-16T00:52:19.151+0000 7f4cfb95f640 -1 mds.l *** got signal Terminated *** 2024-09-16T00:52:23.300 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:52:23 smithi102 podman[70605]: 2024-09-16 00:52:23.023506406 +0000 UTC m=+3.999882943 container died c7327996730350fabecfd8bf3d8d1e113a22bfe9eafcd115fcd117390e194f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T00:52:23.776 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:52:23 smithi102 podman[70605]: 2024-09-16 00:52:23.377416112 +0000 UTC m=+4.353792647 container cleanup c7327996730350fabecfd8bf3d8d1e113a22bfe9eafcd115fcd117390e194f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2) 2024-09-16T00:52:23.776 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:52:23 smithi102 bash[70605]: ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l 2024-09-16T00:52:23.776 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:52:23 smithi102 podman[70619]: 2024-09-16 00:52:23.561651644 +0000 UTC m=+0.533372876 container remove c7327996730350fabecfd8bf3d8d1e113a22bfe9eafcd115fcd117390e194f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-ccb601ea-73c2-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T00:52:23.791 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ccb601ea-73c2-11ef-bceb-c7b262605968@mds.l.service' 2024-09-16T00:52:23.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:52:23.823 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-16T00:52:23.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:52:24.038 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:52:26.548 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:52:26.748 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:52:29.402 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:52:29.599 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:52:31.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:52:32.002 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:52:32.035 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:52:32.062 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T00:52:32.064 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403/remote/smithi031/crash 2024-09-16T00:52:32.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:52:32.095 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:52:32.095 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:52:32.097 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403/remote/smithi081/crash 2024-09-16T00:52:32.098 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:52:32.210 INFO:teuthology.orchestra.run.smithi081.stderr:tar: /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:52:32.210 INFO:teuthology.orchestra.run.smithi081.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:52:32.212 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403/remote/smithi102/crash 2024-09-16T00:52:32.213 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:52:32.254 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:52:32.254 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:52:32.255 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T00:52:32.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-16T00:52:32.300 INFO:tasks.cephadm:Compressing logs... 2024-09-16T00:52:32.300 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:52:32.343 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:52:32.346 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose -- 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.log 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T00:52:32.385 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mon.a.log: 82.6% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.6.log: /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.g.log: 90.2% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.d.log: 86.4% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log.gz 94.7% 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:52:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.3.log: 94.7% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:: No such file or directory 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr: 82.4% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:52:32.387 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-16T00:52:32.388 INFO:teuthology.orchestra.run.smithi102.stderr: 90.4% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:52:32.388 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-16T00:52:32.388 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.l.log: 86.5% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:52:32.388 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-16T00:52:32.388 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.8.log: 94.8% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-16T00:52:32.388 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-16T00:52:32.389 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/cephadm.log: 90.9% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:52:32.391 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mgr.z.log: 90.8% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:52:32.391 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-16T00:52:32.391 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.c.log: 92.8% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-16T00:52:32.391 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:52:32.391 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.f.log: 94.6% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-16T00:52:32.391 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-16T00:52:32.392 INFO:teuthology.orchestra.run.smithi081.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T00:52:32.393 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-16T00:52:32.393 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T00:52:32.393 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T00:52:32.394 INFO:teuthology.orchestra.run.smithi102.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-16T00:52:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T00:52:32.398 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-16T00:52:32.398 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:52:32.398 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-16T00:52:32.398 INFO:teuthology.orchestra.run.smithi031.stderr: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:52:32.398 INFO:teuthology.orchestra.run.smithi081.stderr: 94.7% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-16T00:52:32.399 INFO:teuthology.orchestra.run.smithi081.stderr: 94.7% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-16T00:52:32.428 INFO:teuthology.orchestra.run.smithi081.stderr: 82.5% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:52:32.505 INFO:teuthology.orchestra.run.smithi081.stderr: 86.4% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:52:32.509 INFO:teuthology.orchestra.run.smithi081.stderr: 92.6% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T00:52:32.556 INFO:teuthology.orchestra.run.smithi081.stderr: 90.9% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:52:32.643 INFO:teuthology.orchestra.run.smithi031.stderr: 90.1% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T00:52:32.643 INFO:teuthology.orchestra.run.smithi102.stderr: 91.8% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T00:52:32.742 INFO:teuthology.orchestra.run.smithi081.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:52:32.872 INFO:teuthology.orchestra.run.smithi081.stderr: 90.3% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:52:33.110 INFO:teuthology.orchestra.run.smithi031.stderr: 90.4% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T00:52:33.764 INFO:teuthology.orchestra.run.smithi081.stderr: 91.8% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T00:52:47.362 INFO:teuthology.orchestra.run.smithi031.stderr: 91.8% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T00:52:48.203 INFO:teuthology.orchestra.run.smithi102.stderr: 91.7% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-16T00:52:49.189 INFO:teuthology.orchestra.run.smithi031.stderr: 91.7% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T00:52:49.275 INFO:teuthology.orchestra.run.smithi102.stderr: 91.9% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T00:52:49.713 INFO:teuthology.orchestra.run.smithi031.stderr: 91.9% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T00:52:50.125 INFO:teuthology.orchestra.run.smithi031.stderr: 91.9% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-16T00:52:50.127 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-16T00:52:50.127 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m17.769s 2024-09-16T00:52:50.127 INFO:teuthology.orchestra.run.smithi031.stderr:user 1m5.929s 2024-09-16T00:52:50.127 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m1.929s 2024-09-16T00:52:50.202 INFO:teuthology.orchestra.run.smithi102.stderr: 95.1% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-16T00:52:51.374 INFO:teuthology.orchestra.run.smithi102.stderr: 91.7% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-16T00:52:53.938 INFO:teuthology.orchestra.run.smithi102.stderr: 91.6% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T00:52:53.940 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-09-16T00:52:53.940 INFO:teuthology.orchestra.run.smithi102.stderr:real 0m21.579s 2024-09-16T00:52:53.940 INFO:teuthology.orchestra.run.smithi102.stderr:user 1m28.566s 2024-09-16T00:52:53.940 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m2.717s 2024-09-16T00:53:00.394 INFO:teuthology.orchestra.run.smithi081.stderr: 91.9% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T00:53:37.122 INFO:teuthology.orchestra.run.smithi081.stderr: 91.8% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-16T00:53:48.122 INFO:teuthology.orchestra.run.smithi081.stderr: 91.6% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T00:54:37.855 INFO:teuthology.orchestra.run.smithi081.stderr: 91.9% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T00:58:25.748 INFO:teuthology.orchestra.run.smithi081.stderr: 93.7% -- replaced with /var/log/ceph/ccb601ea-73c2-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-16T00:58:25.752 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-09-16T00:58:25.752 INFO:teuthology.orchestra.run.smithi081.stderr:real 5m53.385s 2024-09-16T00:58:25.752 INFO:teuthology.orchestra.run.smithi081.stderr:user 4m50.831s 2024-09-16T00:58:25.752 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m10.759s 2024-09-16T00:58:25.752 INFO:tasks.cephadm:Archiving logs... 2024-09-16T00:58:25.754 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403/remote/smithi031/log 2024-09-16T00:58:25.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:58:36.384 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403/remote/smithi081/log 2024-09-16T00:58:36.385 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:59:22.425 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403/remote/smithi102/log 2024-09-16T00:59:22.428 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:59:35.792 INFO:tasks.cephadm:Removing cluster... 2024-09-16T00:59:35.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 --force 2024-09-16T00:59:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:59:36.376 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 --force 2024-09-16T00:59:36.708 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:59:37.263 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ccb601ea-73c2-11ef-bceb-c7b262605968 --force 2024-09-16T00:59:37.467 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: ccb601ea-73c2-11ef-bceb-c7b262605968 2024-09-16T00:59:37.837 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T00:59:37.838 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:59:37.858 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:59:37.885 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:59:37.902 INFO:tasks.cephadm:Teardown complete 2024-09-16T00:59:37.902 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T00:59:37.917 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T00:59:37.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:59:37.920 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:59:37.928 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:59:38.042 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T00:59:38.052 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T00:59:38.052 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:59:38.055 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:59:38.058 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:59:38.070 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:59:38.070 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:59:38.072 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:59:38.075 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:59:38.075 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-16T00:59:38.075 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 -145us[ -131us] +/- 44ms 2024-09-16T00:59:38.075 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 -677us[ -664us] +/- 51ms 2024-09-16T00:59:38.075 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +1450us[+1463us] +/- 81ms 2024-09-16T00:59:38.076 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:59:38.077 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:59:38.077 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-16T00:59:38.078 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -147us[ -129us] +/- 44ms 2024-09-16T00:59:38.078 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -678us[ -661us] +/- 51ms 2024-09-16T00:59:38.078 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +1452us[+1452us] +/- 81ms 2024-09-16T00:59:38.078 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:59:38.093 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:59:38.093 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-09-16T00:59:38.093 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 -162us[ -153us] +/- 44ms 2024-09-16T00:59:38.094 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -694us[ -694us] +/- 51ms 2024-09-16T00:59:38.094 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +1426us[+1426us] +/- 81ms 2024-09-16T00:59:38.094 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:59:38.094 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T00:59:38.104 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T00:59:38.104 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T00:59:38.115 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:59:38.205 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:59:38.282 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:59:38.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:59:38.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:59:38.381 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:59:38.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:59:38.409 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:59:38.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:59:38.437 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T00:59:38.448 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T00:59:38.458 INFO:teuthology.task.internal:Duration was 2120.842340 seconds 2024-09-16T00:59:38.458 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T00:59:38.467 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T00:59:38.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:59:38.470 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:59:38.472 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:59:38.510 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:59:38.516 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:59:38.518 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:59:38.868 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T00:59:38.868 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:59:38.868 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:59:38.891 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-09-16T00:59:38.891 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:59:38.923 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T00:59:38.923 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:59:38.951 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T00:59:38.952 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:59:38.954 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:59:38.967 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:59:39.018 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T00:59:39.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:59:39.024 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:59:39.038 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:59:39.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T00:59:39.912 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T00:59:39.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:59:39.938 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:59:40.010 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:59:40.039 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T00:59:40.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:59:40.053 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:59:40.056 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:59:40.074 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-16T00:59:40.101 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-09-16T00:59:40.106 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-09-16T00:59:40.151 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:59:40.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:59:40.167 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:59:40.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:59:40.182 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:59:40.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:59:40.207 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T00:59:40.222 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T00:59:40.223 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403/remote/smithi031 2024-09-16T00:59:40.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:59:40.288 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403/remote/smithi081 2024-09-16T00:59:40.288 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:59:40.324 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906403/remote/smithi102 2024-09-16T00:59:40.324 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:59:40.361 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T00:59:40.362 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:59:40.364 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:59:40.370 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:59:40.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T00:59:40.430 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T00:59:40.431 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T00:59:40.447 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T00:59:40.447 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:59:40.450 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:59:40.452 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:59:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: 265336 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:59 /home/ubuntu/cephtest 2024-09-16T00:59:40.465 INFO:teuthology.orchestra.run.smithi081.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:59 /home/ubuntu/cephtest 2024-09-16T00:59:40.468 INFO:teuthology.orchestra.run.smithi102.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:59 /home/ubuntu/cephtest 2024-09-16T00:59:40.470 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T00:59:40.482 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T00:59:40.505 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2120.842339992523 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T00:59:40.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:59:40.582 INFO:teuthology.run:pass